2024-08-20T21:41:47.1549820Z Current runner version: '2.319.1' 2024-08-20T21:41:47.1555620Z Runner name: 'i-0a9f60968c6bae802' 2024-08-20T21:41:47.1556291Z Runner group name: 'Default' 2024-08-20T21:41:47.1556990Z Machine name: 'ip-10-0-15-34' 2024-08-20T21:41:47.1573894Z Testing runner upgrade compatibility 2024-08-20T21:41:47.3279601Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T21:41:47.3281665Z Actions: read 2024-08-20T21:41:47.3282060Z Attestations: read 2024-08-20T21:41:47.3282472Z Checks: read 2024-08-20T21:41:47.3282828Z Contents: read 2024-08-20T21:41:47.3283187Z Deployments: read 2024-08-20T21:41:47.3283691Z Discussions: read 2024-08-20T21:41:47.3284228Z Issues: read 2024-08-20T21:41:47.3284564Z Metadata: read 2024-08-20T21:41:47.3284953Z Packages: read 2024-08-20T21:41:47.3285275Z Pages: read 2024-08-20T21:41:47.3285620Z PullRequests: read 2024-08-20T21:41:47.3286043Z RepositoryProjects: read 2024-08-20T21:41:47.3286447Z SecurityEvents: read 2024-08-20T21:41:47.3286786Z Statuses: read 2024-08-20T21:41:47.3287390Z ##[endgroup] 2024-08-20T21:41:47.3291028Z Secret source: Actions 2024-08-20T21:41:47.3291735Z Prepare workflow directory 2024-08-20T21:41:47.6640323Z Prepare all required actions 2024-08-20T21:41:47.6801413Z Getting action download info 2024-08-20T21:41:47.8559559Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T21:41:48.1704726Z Download action repository 'pytorch/pytorch@main' (SHA:15b5a0b67fc3f34fb0bf1afa6f91e0c4c2b7fd8d) 2024-08-20T21:41:51.2190920Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T21:41:51.4428574Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T21:41:51.6854525Z Getting action download info 2024-08-20T21:41:51.8451116Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T21:41:52.0268457Z Getting action download info 2024-08-20T21:41:52.1137118Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T21:41:52.2562479Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T21:41:52.2566428Z ##[group] Inputs 2024-08-20T21:41:52.2566934Z build-environment: linux-jammy-py3.8-gcc11-build 2024-08-20T21:41:52.2586918Z test-matrix: {"include": [{"config": "inductor_avx512", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_avx512", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_amp_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_torchbench_cpu_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.24xl.spr-metal"}, {"config": "inductor_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "inductor_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_huggingface_freezing_avx2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}]} 2024-08-20T21:41:52.2607578Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:41:52.2608731Z sync-tag: 2024-08-20T21:41:52.2609676Z timeout-minutes: 240 2024-08-20T21:41:52.2610010Z use-gha: 2024-08-20T21:41:52.2610271Z dashboard-tag: 2024-08-20T21:41:52.2610584Z s3-bucket: gha-artifacts 2024-08-20T21:41:52.2610934Z aws-role-to-assume: 2024-08-20T21:41:52.2611267Z ##[endgroup] 2024-08-20T21:41:52.2612372Z Complete job name: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:41:52.3394966Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T21:41:52.3531549Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T21:41:52.3540651Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:41:52.3541325Z ##[endgroup] 2024-08-20T21:41:53.8745044Z Runner Type: amz2023.linux.12xlarge 2024-08-20T21:41:53.8745534Z Instance Type: c5.12xlarge 2024-08-20T21:41:53.8746194Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T21:41:53.8746699Z AMI ID: ami-06c68f701d8090592 2024-08-20T21:41:59.1891841Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T21:41:59.1892397Z with: 2024-08-20T21:41:59.1893105Z github-secret: *** 2024-08-20T21:41:59.1894070Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T21:41:59.1895113Z activate-with-label: false 2024-08-20T21:41:59.1895447Z label: with-ssh 2024-08-20T21:41:59.1895860Z remove-existing-keys: true 2024-08-20T21:41:59.1896202Z fail-silently: true 2024-08-20T21:41:59.1896475Z env: 2024-08-20T21:41:59.1896726Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:41:59.1897046Z ##[endgroup] 2024-08-20T21:41:59.2697280Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T21:41:59.2698668Z ciflow reference detected, attempting to extract PR number 2024-08-20T21:41:59.5310060Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T21:41:59.6314575Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T21:41:59.6315224Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T21:41:59.6946614Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T21:41:59.6970328Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T21:41:59.6995158Z Login using: ssh ec2-user@ec2-34-230-62-23.compute-1.amazonaws.com 2024-08-20T21:41:59.6995973Z All testing is done inside the container, to start an interactive session run: 2024-08-20T21:41:59.6996838Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T21:41:59.7122975Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T21:41:59.7123835Z with: 2024-08-20T21:41:59.7124114Z submodules: recursive 2024-08-20T21:41:59.7124481Z fetch-depth: 0 2024-08-20T21:41:59.7124765Z env: 2024-08-20T21:41:59.7125033Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:41:59.7125357Z ##[endgroup] 2024-08-20T21:41:59.7311432Z ##[group]Run retry () { 2024-08-20T21:41:59.7311780Z retry () { 2024-08-20T21:41:59.7312246Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T21:41:59.7312768Z } 2024-08-20T21:41:59.7313031Z echo "${GITHUB_WORKSPACE}" 2024-08-20T21:41:59.7313418Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T21:41:59.7313860Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T21:41:59.7314283Z else 2024-08-20T21:41:59.7314579Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T21:41:59.7314971Z fi 2024-08-20T21:41:59.7315302Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T21:41:59.7324450Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:41:59.7324974Z env: 2024-08-20T21:41:59.7325242Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:41:59.7325567Z NO_SUDO: 2024-08-20T21:41:59.7325835Z ##[endgroup] 2024-08-20T21:41:59.7352981Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:42:02.0926936Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T21:42:02.0927351Z with: 2024-08-20T21:42:02.0927664Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:42:02.0928101Z fetch-depth: 0 2024-08-20T21:42:02.0928400Z submodules: recursive 2024-08-20T21:42:02.0928712Z quiet-checkout: true 2024-08-20T21:42:02.0929048Z repository: pytorch/pytorch 2024-08-20T21:42:02.0929529Z token: *** 2024-08-20T21:42:02.0929801Z ssh-strict: true 2024-08-20T21:42:02.0930100Z persist-credentials: true 2024-08-20T21:42:02.0930442Z clean: true 2024-08-20T21:42:02.0930760Z sparse-checkout-cone-mode: true 2024-08-20T21:42:02.0931326Z lfs: false 2024-08-20T21:42:02.0931597Z set-safe-directory: true 2024-08-20T21:42:02.0931920Z env: 2024-08-20T21:42:02.0932172Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:42:02.0932487Z ##[endgroup] 2024-08-20T21:42:02.1840994Z Syncing repository: pytorch/pytorch 2024-08-20T21:42:02.1842610Z ##[group]Getting Git version info 2024-08-20T21:42:02.1843481Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T21:42:02.1844371Z [command]/usr/bin/git version 2024-08-20T21:42:02.1844721Z git version 2.40.1 2024-08-20T21:42:02.1846011Z ##[endgroup] 2024-08-20T21:42:02.1857117Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/58b34222-d97f-4396-99a0-007aa764fa50' before making global git config changes 2024-08-20T21:42:02.1858399Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T21:42:02.1859551Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:42:02.1889774Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T21:42:02.1893100Z ##[group]Initializing the repository 2024-08-20T21:42:02.1895597Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:42:02.1925262Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T21:42:02.1926127Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T21:42:02.1926934Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T21:42:02.1927510Z hint: 2024-08-20T21:42:02.1928090Z hint: git config --global init.defaultBranch 2024-08-20T21:42:02.1928548Z hint: 2024-08-20T21:42:02.1929055Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T21:42:02.1929917Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T21:42:02.1930748Z hint: 2024-08-20T21:42:02.1931063Z hint: git branch -m 2024-08-20T21:42:02.1931810Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T21:42:02.1933608Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T21:42:02.1966217Z ##[endgroup] 2024-08-20T21:42:02.1966773Z ##[group]Disabling automatic garbage collection 2024-08-20T21:42:02.1968752Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T21:42:02.1997502Z ##[endgroup] 2024-08-20T21:42:02.1998006Z ##[group]Setting up auth 2024-08-20T21:42:02.2001597Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T21:42:02.2030336Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T21:42:02.2296349Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T21:42:02.2324430Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T21:42:02.2587461Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T21:42:02.2634417Z ##[endgroup] 2024-08-20T21:42:02.2634999Z ##[group]Fetching the repository 2024-08-20T21:42:02.2638912Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T21:42:05.1746119Z remote: Enumerating objects: 1008510 2024-08-20T21:42:05.1746710Z remote: Enumerating objects: 1010152, done. 2024-08-20T21:42:05.1747386Z remote: Counting objects: 0% (1/1642) 2024-08-20T21:42:05.1747908Z remote: Counting objects: 1% (17/1642) 2024-08-20T21:42:05.1748502Z remote: Counting objects: 2% (33/1642) 2024-08-20T21:42:05.1750155Z remote: Counting objects: 3% (50/1642) 2024-08-20T21:42:05.1750641Z remote: Counting objects: 4% (66/1642) 2024-08-20T21:42:05.1751153Z remote: Counting objects: 5% (83/1642) 2024-08-20T21:42:05.1751702Z remote: Counting objects: 6% (99/1642) 2024-08-20T21:42:05.1752217Z remote: Counting objects: 7% (115/1642) 2024-08-20T21:42:05.1752717Z remote: Counting objects: 8% (132/1642) 2024-08-20T21:42:05.1753231Z remote: Counting objects: 9% (148/1642) 2024-08-20T21:42:05.1753748Z remote: Counting objects: 10% (165/1642) 2024-08-20T21:42:05.1754256Z remote: Counting objects: 11% (181/1642) 2024-08-20T21:42:05.1754747Z remote: Counting objects: 12% (198/1642) 2024-08-20T21:42:05.1755256Z remote: Counting objects: 13% (214/1642) 2024-08-20T21:42:05.1755766Z remote: Counting objects: 14% (230/1642) 2024-08-20T21:42:05.1756281Z remote: Counting objects: 15% (247/1642) 2024-08-20T21:42:05.1756770Z remote: Counting objects: 16% (263/1642) 2024-08-20T21:42:05.1757280Z remote: Counting objects: 17% (280/1642) 2024-08-20T21:42:05.1757784Z remote: Counting objects: 18% (296/1642) 2024-08-20T21:42:05.1758283Z remote: Counting objects: 19% (312/1642) 2024-08-20T21:42:05.1758765Z remote: Counting objects: 20% (329/1642) 2024-08-20T21:42:05.1759267Z remote: Counting objects: 21% (345/1642) 2024-08-20T21:42:05.1759769Z remote: Counting objects: 22% (362/1642) 2024-08-20T21:42:05.1760271Z remote: Counting objects: 23% (378/1642) 2024-08-20T21:42:05.1760748Z remote: Counting objects: 24% (395/1642) 2024-08-20T21:42:05.1761250Z remote: Counting objects: 25% (411/1642) 2024-08-20T21:42:05.1761754Z remote: Counting objects: 26% (427/1642) 2024-08-20T21:42:05.1762255Z remote: Counting objects: 27% (444/1642) 2024-08-20T21:42:05.1762900Z remote: Counting objects: 28% (460/1642) 2024-08-20T21:42:05.1763566Z remote: Counting objects: 29% (477/1642) 2024-08-20T21:42:05.1764362Z remote: Counting objects: 30% (493/1642) 2024-08-20T21:42:05.1764862Z remote: Counting objects: 31% (510/1642) 2024-08-20T21:42:05.1765343Z remote: Counting objects: 32% (526/1642) 2024-08-20T21:42:05.1765849Z remote: Counting objects: 33% (542/1642) 2024-08-20T21:42:05.1766359Z remote: Counting objects: 34% (559/1642) 2024-08-20T21:42:05.1766860Z remote: Counting objects: 35% (575/1642) 2024-08-20T21:42:05.1767343Z remote: Counting objects: 36% (592/1642) 2024-08-20T21:42:05.1767849Z remote: Counting objects: 37% (608/1642) 2024-08-20T21:42:05.1768352Z remote: Counting objects: 38% (624/1642) 2024-08-20T21:42:05.1768855Z remote: Counting objects: 39% (641/1642) 2024-08-20T21:42:05.1769335Z remote: Counting objects: 40% (657/1642) 2024-08-20T21:42:05.1769847Z remote: Counting objects: 41% (674/1642) 2024-08-20T21:42:05.1770470Z remote: Counting objects: 42% (690/1642) 2024-08-20T21:42:05.1770962Z remote: Counting objects: 43% (707/1642) 2024-08-20T21:42:05.1771433Z remote: Counting objects: 44% (723/1642) 2024-08-20T21:42:05.1771921Z remote: Counting objects: 45% (739/1642) 2024-08-20T21:42:05.1772411Z remote: Counting objects: 46% (756/1642) 2024-08-20T21:42:05.1772902Z remote: Counting objects: 47% (772/1642) 2024-08-20T21:42:05.1773369Z remote: Counting objects: 48% (789/1642) 2024-08-20T21:42:05.1773860Z remote: Counting objects: 49% (805/1642) 2024-08-20T21:42:05.1774349Z remote: Counting objects: 50% (821/1642) 2024-08-20T21:42:05.1774822Z remote: Counting objects: 51% (838/1642) 2024-08-20T21:42:05.1775308Z remote: Counting objects: 52% (854/1642) 2024-08-20T21:42:05.1775799Z remote: Counting objects: 53% (871/1642) 2024-08-20T21:42:05.1776294Z remote: Counting objects: 54% (887/1642) 2024-08-20T21:42:05.1776886Z remote: Counting objects: 55% (904/1642) 2024-08-20T21:42:05.1777375Z remote: Counting objects: 56% (920/1642) 2024-08-20T21:42:05.1777868Z remote: Counting objects: 57% (936/1642) 2024-08-20T21:42:05.1778362Z remote: Counting objects: 58% (953/1642) 2024-08-20T21:42:05.1778834Z remote: Counting objects: 59% (969/1642) 2024-08-20T21:42:05.1779325Z remote: Counting objects: 60% (986/1642) 2024-08-20T21:42:05.1779821Z remote: Counting objects: 61% (1002/1642) 2024-08-20T21:42:05.1780336Z remote: Counting objects: 62% (1019/1642) 2024-08-20T21:42:05.1780866Z remote: Counting objects: 63% (1035/1642) 2024-08-20T21:42:05.1781362Z remote: Counting objects: 64% (1051/1642) 2024-08-20T21:42:05.1781901Z remote: Counting objects: 65% (1068/1642) 2024-08-20T21:42:05.1782418Z remote: Counting objects: 66% (1084/1642) 2024-08-20T21:42:05.1782928Z remote: Counting objects: 67% (1101/1642) 2024-08-20T21:42:05.1783440Z remote: Counting objects: 68% (1117/1642) 2024-08-20T21:42:05.1783930Z remote: Counting objects: 69% (1133/1642) 2024-08-20T21:42:05.1784438Z remote: Counting objects: 70% (1150/1642) 2024-08-20T21:42:05.1784946Z remote: Counting objects: 71% (1166/1642) 2024-08-20T21:42:05.1785455Z remote: Counting objects: 72% (1183/1642) 2024-08-20T21:42:05.1785947Z remote: Counting objects: 73% (1199/1642) 2024-08-20T21:42:05.1786451Z remote: Counting objects: 74% (1216/1642) 2024-08-20T21:42:05.1786959Z remote: Counting objects: 75% (1232/1642) 2024-08-20T21:42:05.1787465Z remote: Counting objects: 76% (1248/1642) 2024-08-20T21:42:05.1787956Z remote: Counting objects: 77% (1265/1642) 2024-08-20T21:42:05.1788459Z remote: Counting objects: 78% (1281/1642) 2024-08-20T21:42:05.1789069Z remote: Counting objects: 79% (1298/1642) 2024-08-20T21:42:05.1789585Z remote: Counting objects: 80% (1314/1642) 2024-08-20T21:42:05.1790072Z remote: Counting objects: 81% (1331/1642) 2024-08-20T21:42:05.1790577Z remote: Counting objects: 82% (1347/1642) 2024-08-20T21:42:05.1791085Z remote: Counting objects: 83% (1363/1642) 2024-08-20T21:42:05.1791594Z remote: Counting objects: 84% (1380/1642) 2024-08-20T21:42:05.1792084Z remote: Counting objects: 85% (1396/1642) 2024-08-20T21:42:05.1792596Z remote: Counting objects: 86% (1413/1642) 2024-08-20T21:42:05.1793104Z remote: Counting objects: 87% (1429/1642) 2024-08-20T21:42:05.1793612Z remote: Counting objects: 88% (1445/1642) 2024-08-20T21:42:05.1794102Z remote: Counting objects: 89% (1462/1642) 2024-08-20T21:42:05.1794605Z remote: Counting objects: 90% (1478/1642) 2024-08-20T21:42:05.1795111Z remote: Counting objects: 91% (1495/1642) 2024-08-20T21:42:05.1795626Z remote: Counting objects: 92% (1511/1642) 2024-08-20T21:42:05.1796120Z remote: Counting objects: 93% (1528/1642) 2024-08-20T21:42:05.1796626Z remote: Counting objects: 94% (1544/1642) 2024-08-20T21:42:05.1797131Z remote: Counting objects: 95% (1560/1642) 2024-08-20T21:42:05.1797636Z remote: Counting objects: 96% (1577/1642) 2024-08-20T21:42:05.1798128Z remote: Counting objects: 97% (1593/1642) 2024-08-20T21:42:05.1798634Z remote: Counting objects: 98% (1610/1642) 2024-08-20T21:42:05.1799138Z remote: Counting objects: 99% (1626/1642) 2024-08-20T21:42:05.1799626Z remote: Counting objects: 100% (1642/1642) 2024-08-20T21:42:05.1800167Z remote: Counting objects: 100% (1642/1642), done. 2024-08-20T21:42:05.1951132Z remote: Compressing objects: 0% (1/835) 2024-08-20T21:42:05.2212575Z remote: Compressing objects: 1% (9/835) 2024-08-20T21:42:05.2608123Z remote: Compressing objects: 2% (17/835) 2024-08-20T21:42:05.3252366Z remote: Compressing objects: 3% (26/835) 2024-08-20T21:42:05.4005196Z remote: Compressing objects: 4% (34/835) 2024-08-20T21:42:05.4478508Z remote: Compressing objects: 5% (42/835) 2024-08-20T21:42:05.4944951Z remote: Compressing objects: 6% (51/835) 2024-08-20T21:42:05.5298877Z remote: Compressing objects: 7% (59/835) 2024-08-20T21:42:05.5637064Z remote: Compressing objects: 8% (67/835) 2024-08-20T21:42:05.5945044Z remote: Compressing objects: 9% (76/835) 2024-08-20T21:42:05.6162415Z remote: Compressing objects: 10% (84/835) 2024-08-20T21:42:05.6389814Z remote: Compressing objects: 11% (92/835) 2024-08-20T21:42:05.6606351Z remote: Compressing objects: 12% (101/835) 2024-08-20T21:42:05.6752892Z remote: Compressing objects: 13% (109/835) 2024-08-20T21:42:05.6862584Z remote: Compressing objects: 14% (117/835) 2024-08-20T21:42:05.6930491Z remote: Compressing objects: 15% (126/835) 2024-08-20T21:42:05.7022075Z remote: Compressing objects: 16% (134/835) 2024-08-20T21:42:05.7148201Z remote: Compressing objects: 17% (142/835) 2024-08-20T21:42:05.7149048Z remote: Compressing objects: 18% (151/835) 2024-08-20T21:42:05.7191481Z remote: Compressing objects: 19% (159/835) 2024-08-20T21:42:05.7228741Z remote: Compressing objects: 20% (167/835) 2024-08-20T21:42:05.7253039Z remote: Compressing objects: 21% (176/835) 2024-08-20T21:42:05.7267446Z remote: Compressing objects: 22% (184/835) 2024-08-20T21:42:05.7268965Z remote: Compressing objects: 23% (193/835) 2024-08-20T21:42:05.7275063Z remote: Compressing objects: 24% (201/835) 2024-08-20T21:42:05.7284847Z remote: Compressing objects: 25% (209/835) 2024-08-20T21:42:05.7288667Z remote: Compressing objects: 26% (218/835) 2024-08-20T21:42:05.7306826Z remote: Compressing objects: 27% (226/835) 2024-08-20T21:42:05.7309795Z remote: Compressing objects: 28% (234/835) 2024-08-20T21:42:05.7337531Z remote: Compressing objects: 29% (243/835) 2024-08-20T21:42:05.7339144Z remote: Compressing objects: 30% (251/835) 2024-08-20T21:42:05.7355175Z remote: Compressing objects: 31% (259/835) 2024-08-20T21:42:05.7363095Z remote: Compressing objects: 32% (268/835) 2024-08-20T21:42:05.7366396Z remote: Compressing objects: 33% (276/835) 2024-08-20T21:42:05.7386856Z remote: Compressing objects: 34% (284/835) 2024-08-20T21:42:05.7387454Z remote: Compressing objects: 35% (293/835) 2024-08-20T21:42:05.7395347Z remote: Compressing objects: 36% (301/835) 2024-08-20T21:42:05.7400764Z remote: Compressing objects: 37% (309/835) 2024-08-20T21:42:05.7410600Z remote: Compressing objects: 38% (318/835) 2024-08-20T21:42:05.7418900Z remote: Compressing objects: 39% (326/835) 2024-08-20T21:42:05.7434879Z remote: Compressing objects: 40% (334/835) 2024-08-20T21:42:05.7435574Z remote: Compressing objects: 41% (343/835) 2024-08-20T21:42:05.7444508Z remote: Compressing objects: 42% (351/835) 2024-08-20T21:42:05.7446481Z remote: Compressing objects: 43% (360/835) 2024-08-20T21:42:05.7454590Z remote: Compressing objects: 44% (368/835) 2024-08-20T21:42:05.7460401Z remote: Compressing objects: 45% (376/835) 2024-08-20T21:42:05.7465989Z remote: Compressing objects: 46% (385/835) 2024-08-20T21:42:05.7472971Z remote: Compressing objects: 47% (393/835) 2024-08-20T21:42:05.7473513Z remote: Compressing objects: 48% (401/835) 2024-08-20T21:42:05.7477227Z remote: Compressing objects: 49% (410/835) 2024-08-20T21:42:05.7481751Z remote: Compressing objects: 50% (418/835) 2024-08-20T21:42:05.7485009Z remote: Compressing objects: 51% (426/835) 2024-08-20T21:42:05.7489555Z remote: Compressing objects: 52% (435/835) 2024-08-20T21:42:05.7493015Z remote: Compressing objects: 53% (443/835) 2024-08-20T21:42:05.7495494Z remote: Compressing objects: 54% (451/835) 2024-08-20T21:42:05.7498512Z remote: Compressing objects: 55% (460/835) 2024-08-20T21:42:05.7500842Z remote: Compressing objects: 56% (468/835) 2024-08-20T21:42:05.7502982Z remote: Compressing objects: 57% (476/835) 2024-08-20T21:42:05.7504297Z remote: Compressing objects: 58% (485/835) 2024-08-20T21:42:05.7505897Z remote: Compressing objects: 59% (493/835) 2024-08-20T21:42:05.7507165Z remote: Compressing objects: 60% (501/835) 2024-08-20T21:42:05.7508585Z remote: Compressing objects: 61% (510/835) 2024-08-20T21:42:05.7509273Z remote: Compressing objects: 62% (518/835) 2024-08-20T21:42:05.7509809Z remote: Compressing objects: 63% (527/835) 2024-08-20T21:42:05.7511276Z remote: Compressing objects: 64% (535/835) 2024-08-20T21:42:05.7514317Z remote: Compressing objects: 65% (543/835) 2024-08-20T21:42:05.7521831Z remote: Compressing objects: 66% (552/835) 2024-08-20T21:42:05.7525680Z remote: Compressing objects: 67% (560/835) 2024-08-20T21:42:05.7531745Z remote: Compressing objects: 68% (568/835) 2024-08-20T21:42:05.7536068Z remote: Compressing objects: 69% (577/835) 2024-08-20T21:42:05.7542501Z remote: Compressing objects: 70% (585/835) 2024-08-20T21:42:05.7545447Z remote: Compressing objects: 71% (593/835) 2024-08-20T21:42:05.7549256Z remote: Compressing objects: 72% (602/835) 2024-08-20T21:42:05.7553002Z remote: Compressing objects: 73% (610/835) 2024-08-20T21:42:05.7558875Z remote: Compressing objects: 74% (618/835) 2024-08-20T21:42:05.7561032Z remote: Compressing objects: 75% (627/835) 2024-08-20T21:42:05.7564735Z remote: Compressing objects: 76% (635/835) 2024-08-20T21:42:05.7570148Z remote: Compressing objects: 77% (643/835) 2024-08-20T21:42:05.7572382Z remote: Compressing objects: 78% (652/835) 2024-08-20T21:42:05.7574521Z remote: Compressing objects: 79% (660/835) 2024-08-20T21:42:05.7577628Z remote: Compressing objects: 80% (668/835) 2024-08-20T21:42:05.7580558Z remote: Compressing objects: 81% (677/835) 2024-08-20T21:42:05.7584778Z remote: Compressing objects: 82% (685/835) 2024-08-20T21:42:05.7587892Z remote: Compressing objects: 83% (694/835) 2024-08-20T21:42:05.7590133Z remote: Compressing objects: 84% (702/835) 2024-08-20T21:42:05.7592012Z remote: Compressing objects: 85% (710/835) 2024-08-20T21:42:05.7594252Z remote: Compressing objects: 86% (719/835) 2024-08-20T21:42:05.7595744Z remote: Compressing objects: 87% (727/835) 2024-08-20T21:42:05.7598168Z remote: Compressing objects: 88% (735/835) 2024-08-20T21:42:05.7600630Z remote: Compressing objects: 89% (744/835) 2024-08-20T21:42:05.7604022Z remote: Compressing objects: 90% (752/835) 2024-08-20T21:42:05.7606908Z remote: Compressing objects: 91% (760/835) 2024-08-20T21:42:05.7608105Z remote: Compressing objects: 92% (769/835) 2024-08-20T21:42:05.7608640Z remote: Compressing objects: 93% (777/835) 2024-08-20T21:42:05.7611879Z remote: Compressing objects: 94% (785/835) 2024-08-20T21:42:05.7612832Z remote: Compressing objects: 95% (794/835) 2024-08-20T21:42:05.7614279Z remote: Compressing objects: 96% (802/835) 2024-08-20T21:42:05.7615370Z remote: Compressing objects: 97% (810/835) 2024-08-20T21:42:05.7617769Z remote: Compressing objects: 98% (819/835) 2024-08-20T21:42:05.7620001Z remote: Compressing objects: 99% (827/835) 2024-08-20T21:42:05.7621251Z remote: Compressing objects: 100% (835/835) 2024-08-20T21:42:05.7621806Z remote: Compressing objects: 100% (835/835), done. 2024-08-20T21:42:25.8241105Z remote: Total 1010152 (delta 1073), reused 1263 (delta 804), pack-reused 1008510 (from 1) 2024-08-20T21:42:47.0504113Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T21:42:47.0531908Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:42:47.0535960Z ##[endgroup] 2024-08-20T21:42:47.0536574Z ##[group]Determining the checkout info 2024-08-20T21:42:47.0537200Z ##[endgroup] 2024-08-20T21:42:47.0537701Z ##[group]Checking out the ref 2024-08-20T21:42:47.0540179Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:42:48.3736117Z ##[endgroup] 2024-08-20T21:42:48.3736713Z ##[group]Setting up auth for fetching submodules 2024-08-20T21:42:48.3739011Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T21:42:48.3797182Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T21:42:48.3826832Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T21:42:48.3856350Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T21:42:48.3885034Z ##[endgroup] 2024-08-20T21:42:48.3885544Z ##[group]Fetching submodules 2024-08-20T21:42:48.3887423Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T21:42:48.4180258Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T21:42:48.4467469Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T21:42:48.4468965Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T21:42:48.4470416Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T21:42:48.4471909Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T21:42:48.4474995Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T21:42:48.4477176Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T21:42:48.4479384Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T21:42:48.4482063Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T21:42:48.4484958Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T21:42:48.4487853Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T21:42:48.4490767Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T21:42:48.4493798Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T21:42:48.4496941Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T21:42:48.4500308Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T21:42:48.4503435Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T21:42:48.4507031Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:42:48.4513223Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T21:42:48.4516434Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T21:42:48.4519991Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T21:42:48.4524098Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T21:42:48.4527962Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T21:42:48.4531966Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T21:42:48.4535956Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T21:42:48.4540127Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T21:42:48.4544301Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T21:42:48.4549021Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T21:42:48.4553616Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T21:42:48.4558234Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T21:42:48.4562887Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T21:42:48.4568310Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T21:42:48.4572955Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T21:42:48.4578041Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T21:42:48.4582836Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T21:42:48.4589367Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T21:42:48.4617347Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T21:42:48.7515837Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T21:42:48.9364843Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T21:42:49.0956281Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T21:42:49.3287039Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T21:42:50.9277500Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T21:42:59.5849081Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T21:42:59.9412896Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T21:43:00.3505062Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T21:43:01.0108230Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T21:43:02.1195604Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T21:43:03.9027074Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T21:43:08.7447641Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T21:43:09.9254432Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T21:43:11.3476492Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T21:43:12.2835501Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T21:43:12.6847371Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T21:43:12.9975552Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T21:43:13.8433384Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T21:43:14.1662468Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T21:43:14.3955195Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T21:43:15.7907378Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T21:43:16.4084806Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T21:43:17.0535616Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T21:43:23.4327804Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T21:43:24.9667219Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T21:43:27.8827210Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T21:43:28.1342488Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T21:43:35.5976626Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T21:43:35.8142802Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T21:43:36.0273865Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T21:43:36.8130518Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T21:43:37.0978731Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T21:43:37.6593144Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T21:43:38.0460446Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T21:43:38.0571873Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T21:43:38.0659006Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T21:43:38.0882130Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T21:43:38.1234634Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T21:43:39.0190539Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T21:43:39.0408443Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T21:43:39.0829435Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T21:43:39.1719357Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T21:43:39.2030422Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T21:43:39.6683301Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T21:43:39.9032888Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T21:43:39.9790407Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T21:43:39.9805574Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:43:39.9808571Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:43:39.9810158Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:43:39.9811745Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:43:39.9814092Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:43:39.9841420Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T21:43:40.8347628Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T21:43:41.3903077Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T21:43:43.1800098Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T21:43:44.0331991Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T21:43:44.3605746Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T21:43:44.4507851Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T21:43:44.8249011Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T21:43:44.8824746Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T21:43:44.8938514Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T21:43:45.0021602Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T21:43:45.0406430Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T21:43:45.0772684Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T21:43:45.1008195Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T21:43:45.1437153Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T21:43:45.1563807Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T21:43:45.1579542Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T21:43:45.1604926Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T21:43:55.9989411Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T21:43:56.0158469Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T21:43:56.0970257Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T21:43:56.0988183Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:43:56.0990655Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:43:56.0992480Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:43:56.1017854Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T21:43:56.6060801Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T21:43:57.6216333Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T21:43:58.5604685Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T21:43:58.5622811Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:43:58.5624626Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:43:58.5626412Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:43:58.5628224Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:43:58.5630231Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:43:58.5633076Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:43:58.5635702Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:43:58.5638524Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:43:58.5666904Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T21:43:59.3698451Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T21:43:59.7020622Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T21:44:00.6553617Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T21:44:00.9419585Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T21:44:01.4267341Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T21:44:02.2735973Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T21:44:07.2836910Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T21:44:07.6424657Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T21:44:07.6601032Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T21:44:07.6936244Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T21:44:07.7067030Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T21:44:07.7084632Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:07.7108793Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T21:44:08.0249072Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T21:44:08.0426474Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T21:44:08.0806267Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T21:44:08.1717642Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T21:44:08.1867273Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T21:44:08.2234610Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T21:44:08.2758979Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T21:44:08.3102377Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T21:44:08.3321035Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T21:44:08.4276825Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T21:44:08.7527813Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T21:44:08.7562561Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:08.7564533Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:08.7591830Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T21:44:09.1801393Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T21:44:09.9420770Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T21:44:09.9727666Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T21:44:10.0355539Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T21:44:10.0372409Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:10.0374404Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:10.0376935Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:10.0379342Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:10.0381992Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:10.0384319Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:10.0386917Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:10.0389311Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:10.0414298Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T21:44:10.4718037Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T21:44:11.3221091Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T21:44:11.6224288Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T21:44:17.5339753Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T21:44:17.8232757Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T21:44:18.0045023Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T21:44:18.3340698Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T21:44:23.8323709Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T21:44:23.8705279Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T21:44:23.8849646Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T21:44:23.9753949Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T21:44:23.9874387Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T21:44:24.0007181Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T21:44:24.0152475Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T21:44:24.0167671Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:24.0169846Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:24.0195923Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T21:44:25.5521593Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T21:44:26.6158058Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T21:44:26.6588880Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T21:44:27.0928931Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T21:44:27.1040316Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T21:44:27.3500346Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T21:44:27.3521582Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:27.3524484Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:27.3550171Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T21:44:27.7622108Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T21:44:28.6183483Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T21:44:28.6838765Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T21:44:28.6923990Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T21:44:28.7040089Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T21:44:28.7375472Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T21:44:28.7648774Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T21:44:28.8028432Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T21:44:28.8267521Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T21:44:28.8282553Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:28.8284512Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:28.8286547Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:28.8288894Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:28.8314485Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T21:44:29.7039894Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T21:44:29.9201666Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T21:44:30.9074031Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T21:44:31.7244114Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T21:44:31.7387457Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T21:44:31.8011374Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T21:44:31.8270677Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T21:44:31.8284347Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:31.8310735Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T21:44:32.0708805Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T21:44:32.0750359Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T21:44:32.1074102Z Entering 'android/libs/fbjni' 2024-08-20T21:44:32.1114925Z Entering 'third_party/FP16' 2024-08-20T21:44:32.1157569Z Entering 'third_party/FXdiv' 2024-08-20T21:44:32.1201644Z Entering 'third_party/NNPACK' 2024-08-20T21:44:32.1242619Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:32.1282762Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:32.1341076Z Entering 'third_party/benchmark' 2024-08-20T21:44:32.1380991Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:32.1420524Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:32.1460744Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:32.1502270Z Entering 'third_party/cutlass' 2024-08-20T21:44:32.1551558Z Entering 'third_party/eigen' 2024-08-20T21:44:32.1595208Z Entering 'third_party/fbgemm' 2024-08-20T21:44:32.1635513Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:32.1675286Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:32.1717505Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:32.1768207Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:32.1806055Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:32.1844869Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:32.1886481Z Entering 'third_party/fmt' 2024-08-20T21:44:32.1925609Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:32.1966040Z Entering 'third_party/gloo' 2024-08-20T21:44:32.2004192Z Entering 'third_party/googletest' 2024-08-20T21:44:32.2043040Z Entering 'third_party/ideep' 2024-08-20T21:44:32.2080007Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:32.2125342Z Entering 'third_party/ittapi' 2024-08-20T21:44:32.2164261Z Entering 'third_party/kineto' 2024-08-20T21:44:32.2203375Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:32.2242355Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:32.2282388Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:32.2321365Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:32.2359661Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:32.2398594Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:32.2439614Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:32.2478666Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:32.2517882Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:32.2556339Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:32.2597562Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:32.2636601Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:32.2675990Z Entering 'third_party/mimalloc' 2024-08-20T21:44:32.2716830Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:32.2756150Z Entering 'third_party/nlohmann' 2024-08-20T21:44:32.2796522Z Entering 'third_party/onnx' 2024-08-20T21:44:32.2849153Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:32.2886449Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:32.2936421Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:32.2979517Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:32.3017753Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:32.3055393Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:32.3095959Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:32.3135646Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:32.3172740Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:32.3209562Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:32.3246978Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:32.3286886Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:32.3331101Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:32.3390193Z Entering 'third_party/pocketfft' 2024-08-20T21:44:32.3428070Z Entering 'third_party/protobuf' 2024-08-20T21:44:32.3467975Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:32.3515658Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:32.3559079Z Entering 'third_party/psimd' 2024-08-20T21:44:32.3603469Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:32.3644205Z Entering 'third_party/pybind11' 2024-08-20T21:44:32.3683200Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:32.3723816Z Entering 'third_party/sleef' 2024-08-20T21:44:32.3762625Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:32.3802605Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:32.3840665Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:32.3880122Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:32.3917443Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:32.3954118Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:32.4011033Z ##[endgroup] 2024-08-20T21:44:32.4013322Z ##[group]Persisting credentials for submodules 2024-08-20T21:44:32.4017368Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T21:44:32.4306323Z Entering 'android/libs/fbjni' 2024-08-20T21:44:32.4362575Z Entering 'third_party/FP16' 2024-08-20T21:44:32.4416301Z Entering 'third_party/FXdiv' 2024-08-20T21:44:32.4466143Z Entering 'third_party/NNPACK' 2024-08-20T21:44:32.4517871Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:32.4566622Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:32.4634818Z Entering 'third_party/benchmark' 2024-08-20T21:44:32.4683799Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:32.4736658Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:32.4788384Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:32.4839664Z Entering 'third_party/cutlass' 2024-08-20T21:44:32.4904250Z Entering 'third_party/eigen' 2024-08-20T21:44:32.4958324Z Entering 'third_party/fbgemm' 2024-08-20T21:44:32.5016919Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:32.5066121Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:32.5117490Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:32.5181902Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:32.5232014Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:32.5282192Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:32.5335716Z Entering 'third_party/fmt' 2024-08-20T21:44:32.5385899Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:32.5437162Z Entering 'third_party/gloo' 2024-08-20T21:44:32.5488301Z Entering 'third_party/googletest' 2024-08-20T21:44:32.5540762Z Entering 'third_party/ideep' 2024-08-20T21:44:32.5597163Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:32.5666853Z Entering 'third_party/ittapi' 2024-08-20T21:44:32.5721877Z Entering 'third_party/kineto' 2024-08-20T21:44:32.5775610Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:32.5824932Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:32.5878191Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:32.5927713Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:32.5979743Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:32.6030858Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:32.6093837Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:32.6143411Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:32.6197560Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:32.6246922Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:32.6300966Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:32.6350133Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:32.6403940Z Entering 'third_party/mimalloc' 2024-08-20T21:44:32.6453044Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:32.6507453Z Entering 'third_party/nlohmann' 2024-08-20T21:44:32.6558753Z Entering 'third_party/onnx' 2024-08-20T21:44:32.6623853Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:32.6684775Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:32.6742561Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:32.6795578Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:32.6843128Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:32.6897023Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:32.6945386Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:32.6995628Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:32.7045140Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:32.7096568Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:32.7146185Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:32.7203387Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:32.7259256Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:32.7327194Z Entering 'third_party/pocketfft' 2024-08-20T21:44:32.7379239Z Entering 'third_party/protobuf' 2024-08-20T21:44:32.7432313Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:32.7482474Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:32.7532522Z Entering 'third_party/psimd' 2024-08-20T21:44:32.7583597Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:32.7634233Z Entering 'third_party/pybind11' 2024-08-20T21:44:32.7682808Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:32.7732576Z Entering 'third_party/sleef' 2024-08-20T21:44:32.7783040Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:32.7836560Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:32.7884141Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:32.7935734Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:32.7985390Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:32.8034517Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:32.8098689Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T21:44:32.8385655Z Entering 'android/libs/fbjni' 2024-08-20T21:44:32.8432917Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T21:44:32.8446339Z Entering 'third_party/FP16' 2024-08-20T21:44:32.8494083Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T21:44:32.8510144Z Entering 'third_party/FXdiv' 2024-08-20T21:44:32.8557613Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T21:44:32.8571019Z Entering 'third_party/NNPACK' 2024-08-20T21:44:32.8618592Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T21:44:32.8635506Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:32.8683075Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T21:44:32.8699509Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:32.8748698Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T21:44:32.8781589Z Entering 'third_party/benchmark' 2024-08-20T21:44:32.8833886Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:44:32.8847751Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:32.8896240Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T21:44:32.8912698Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:32.8957148Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T21:44:32.8972366Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:32.9019908Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T21:44:32.9036709Z Entering 'third_party/cutlass' 2024-08-20T21:44:32.9082941Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T21:44:32.9107732Z Entering 'third_party/eigen' 2024-08-20T21:44:32.9151916Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T21:44:32.9168618Z Entering 'third_party/fbgemm' 2024-08-20T21:44:32.9219141Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T21:44:32.9235570Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:32.9282611Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T21:44:32.9298022Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:32.9343878Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T21:44:32.9358917Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:32.9405409Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T21:44:32.9428613Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:32.9473084Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:32.9486347Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:32.9535340Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T21:44:32.9553420Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:32.9598143Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T21:44:32.9617324Z Entering 'third_party/fmt' 2024-08-20T21:44:32.9665185Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T21:44:32.9681965Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:32.9726008Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T21:44:32.9745784Z Entering 'third_party/gloo' 2024-08-20T21:44:32.9794627Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T21:44:32.9808405Z Entering 'third_party/googletest' 2024-08-20T21:44:32.9856018Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:32.9870464Z Entering 'third_party/ideep' 2024-08-20T21:44:32.9920136Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T21:44:32.9935086Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:32.9981364Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T21:44:33.0003414Z Entering 'third_party/ittapi' 2024-08-20T21:44:33.0054743Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T21:44:33.0070595Z Entering 'third_party/kineto' 2024-08-20T21:44:33.0117193Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T21:44:33.0132792Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:33.0179913Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T21:44:33.0195217Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:33.0242554Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T21:44:33.0259621Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:33.0309431Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T21:44:33.0323019Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:33.0369884Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T21:44:33.0386769Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:33.0434848Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T21:44:33.0447536Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:33.0498053Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T21:44:33.0515772Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:33.0562297Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T21:44:33.0575363Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:33.0624720Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:33.0641310Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:33.0687192Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T21:44:33.0705382Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:33.0751728Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T21:44:33.0767341Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:33.0811802Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T21:44:33.0828030Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:33.0875000Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T21:44:33.0888594Z Entering 'third_party/mimalloc' 2024-08-20T21:44:33.0936202Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T21:44:33.0952557Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:33.0998769Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T21:44:33.1016090Z Entering 'third_party/nlohmann' 2024-08-20T21:44:33.1063204Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T21:44:33.1080637Z Entering 'third_party/onnx' 2024-08-20T21:44:33.1131003Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T21:44:33.1161177Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:33.1204576Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:44:33.1220803Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:33.1266516Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:44:33.1289987Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:33.1338707Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T21:44:33.1355648Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:33.1401620Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:44:33.1418104Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:33.1465082Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:33.1480695Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:33.1525682Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T21:44:33.1541127Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:33.1595285Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T21:44:33.1612295Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:33.1659544Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T21:44:33.1672663Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:33.1719779Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T21:44:33.1736000Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:33.1784545Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T21:44:33.1800288Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:33.1844598Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T21:44:33.1862991Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:33.1911361Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T21:44:33.1926636Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:33.1972352Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T21:44:33.2005213Z Entering 'third_party/pocketfft' 2024-08-20T21:44:33.2052842Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T21:44:33.2067419Z Entering 'third_party/protobuf' 2024-08-20T21:44:33.2116536Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T21:44:33.2136378Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:33.2182739Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:44:33.2198442Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:33.2243774Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:33.2263281Z Entering 'third_party/psimd' 2024-08-20T21:44:33.2309011Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T21:44:33.2323114Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:33.2368644Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T21:44:33.2386130Z Entering 'third_party/pybind11' 2024-08-20T21:44:33.2432495Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:44:33.2447008Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:33.2495117Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T21:44:33.2510857Z Entering 'third_party/sleef' 2024-08-20T21:44:33.2555343Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T21:44:33.2570715Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:33.2619376Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T21:44:33.2635018Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:33.2680140Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:33.2696491Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:33.2742083Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T21:44:33.2757570Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:33.2803743Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T21:44:33.2819912Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:33.2866301Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:44:33.2878898Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:33.2925338Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T21:44:33.3643080Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T21:44:33.3925868Z Entering 'android/libs/fbjni' 2024-08-20T21:44:33.3965808Z Entering 'third_party/FP16' 2024-08-20T21:44:33.4003817Z Entering 'third_party/FXdiv' 2024-08-20T21:44:33.4043030Z Entering 'third_party/NNPACK' 2024-08-20T21:44:33.4082443Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:33.4122753Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:33.4175489Z Entering 'third_party/benchmark' 2024-08-20T21:44:33.4218280Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:33.4257210Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:33.4298666Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:33.4337657Z Entering 'third_party/cutlass' 2024-08-20T21:44:33.4385593Z Entering 'third_party/eigen' 2024-08-20T21:44:33.4426635Z Entering 'third_party/fbgemm' 2024-08-20T21:44:33.4465070Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:33.4503233Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:33.4540921Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:33.4586127Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:33.4623531Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:33.4662024Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:33.4704543Z Entering 'third_party/fmt' 2024-08-20T21:44:33.4742235Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:33.4782303Z Entering 'third_party/gloo' 2024-08-20T21:44:33.4822092Z Entering 'third_party/googletest' 2024-08-20T21:44:33.4860539Z Entering 'third_party/ideep' 2024-08-20T21:44:33.4897860Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:33.4944731Z Entering 'third_party/ittapi' 2024-08-20T21:44:33.4985038Z Entering 'third_party/kineto' 2024-08-20T21:44:33.5023524Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:33.5062007Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:33.5103005Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:33.5140871Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:33.5179471Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:33.5217599Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:33.5259033Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:33.5298134Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:33.5337282Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:33.5378968Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:33.5425900Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:33.5463804Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:33.5503985Z Entering 'third_party/mimalloc' 2024-08-20T21:44:33.5542593Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:33.5581749Z Entering 'third_party/nlohmann' 2024-08-20T21:44:33.5622733Z Entering 'third_party/onnx' 2024-08-20T21:44:33.5674638Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:33.5722575Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:33.5764689Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:33.5805425Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:33.5842852Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:33.5882511Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:33.5923090Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:33.5961456Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:33.5999784Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:33.6042006Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:33.6080618Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:33.6119536Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:33.6164210Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:33.6222125Z Entering 'third_party/pocketfft' 2024-08-20T21:44:33.6260443Z Entering 'third_party/protobuf' 2024-08-20T21:44:33.6302285Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:33.6339828Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:33.6380984Z Entering 'third_party/psimd' 2024-08-20T21:44:33.6418717Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:33.6456457Z Entering 'third_party/pybind11' 2024-08-20T21:44:33.6497156Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:33.6535794Z Entering 'third_party/sleef' 2024-08-20T21:44:33.6574093Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:33.6614050Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:33.6651169Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:33.6688520Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:33.6731377Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:33.6767731Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:33.6820432Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T21:44:33.7103412Z Entering 'android/libs/fbjni' 2024-08-20T21:44:33.7141253Z Entering 'third_party/FP16' 2024-08-20T21:44:33.7180520Z Entering 'third_party/FXdiv' 2024-08-20T21:44:33.7219066Z Entering 'third_party/NNPACK' 2024-08-20T21:44:33.7258643Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:33.7298771Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:33.7353938Z Entering 'third_party/benchmark' 2024-08-20T21:44:33.7393880Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:33.7433055Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:33.7471881Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:33.7511780Z Entering 'third_party/cutlass' 2024-08-20T21:44:33.7559871Z Entering 'third_party/eigen' 2024-08-20T21:44:33.7603425Z Entering 'third_party/fbgemm' 2024-08-20T21:44:33.7642481Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:33.7681141Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:33.7719321Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:33.7766702Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:33.7803720Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:33.7842929Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:33.7883065Z Entering 'third_party/fmt' 2024-08-20T21:44:33.7922434Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:33.7962351Z Entering 'third_party/gloo' 2024-08-20T21:44:33.8002279Z Entering 'third_party/googletest' 2024-08-20T21:44:33.8041843Z Entering 'third_party/ideep' 2024-08-20T21:44:33.8079919Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:33.8128222Z Entering 'third_party/ittapi' 2024-08-20T21:44:33.8166235Z Entering 'third_party/kineto' 2024-08-20T21:44:33.8204587Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:33.8242833Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:33.8283222Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:33.8323978Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:33.8362706Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:33.8400536Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:33.8442724Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:33.8484446Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:33.8523562Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:33.8566297Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:33.8610794Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:33.8648452Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:33.8687065Z Entering 'third_party/mimalloc' 2024-08-20T21:44:33.8724812Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:33.8762858Z Entering 'third_party/nlohmann' 2024-08-20T21:44:33.8805636Z Entering 'third_party/onnx' 2024-08-20T21:44:33.8860294Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:33.8899831Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:33.8940692Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:33.8981145Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:33.9018859Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:33.9056674Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:33.9097254Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:33.9139329Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:33.9187793Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:33.9224535Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:33.9261416Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:33.9301819Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:33.9343872Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:33.9399732Z Entering 'third_party/pocketfft' 2024-08-20T21:44:33.9438567Z Entering 'third_party/protobuf' 2024-08-20T21:44:33.9481014Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:33.9519209Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:33.9559523Z Entering 'third_party/psimd' 2024-08-20T21:44:33.9598967Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:33.9642506Z Entering 'third_party/pybind11' 2024-08-20T21:44:33.9681178Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:33.9720329Z Entering 'third_party/sleef' 2024-08-20T21:44:33.9759015Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:33.9803357Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:33.9843609Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:33.9881051Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:33.9919731Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:33.9956074Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:34.0008502Z ##[endgroup] 2024-08-20T21:44:34.0044957Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T21:44:34.0071508Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T21:44:34.0254272Z Prepare all required actions 2024-08-20T21:44:34.0254714Z Getting action download info 2024-08-20T21:44:34.1534270Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T21:44:34.1534646Z env: 2024-08-20T21:44:34.1534896Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:34.1535212Z ##[endgroup] 2024-08-20T21:44:34.1590988Z ##[group]Run set -euo pipefail 2024-08-20T21:44:34.1591527Z set -euo pipefail 2024-08-20T21:44:34.1591867Z function get_ec2_metadata() { 2024-08-20T21:44:34.1592359Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T21:44:34.1593207Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T21:44:34.1593936Z  category=$1 2024-08-20T21:44:34.1594407Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T21:44:34.1594993Z  runner_name_str=i-0a9f60968c6bae802 2024-08-20T21:44:34.1595465Z  if [[ -f /.inarc ]]; then 2024-08-20T21:44:34.1595913Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T21:44:34.1596416Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T21:44:34.1597051Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T21:44:34.1597619Z  else 2024-08-20T21:44:34.1598075Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T21:44:34.1598597Z  fi 2024-08-20T21:44:34.1598843Z } 2024-08-20T21:44:34.1599159Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T21:44:34.1599692Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T21:44:34.1600284Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T21:44:34.1600915Z echo "system info $(uname -a)" 2024-08-20T21:44:34.1607729Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:34.1608224Z env: 2024-08-20T21:44:34.1608483Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:34.1608822Z ##[endgroup] 2024-08-20T21:44:34.1701757Z ami-id: ami-06c68f701d8090592 2024-08-20T21:44:34.1757121Z instance-id: i-0a9f60968c6bae802 2024-08-20T21:44:34.1805183Z instance-type: c5.12xlarge 2024-08-20T21:44:34.1816199Z system info Linux ip-10-0-15-34.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T21:44:34.1836656Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T21:44:34.1837562Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T21:44:34.1844234Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:34.1844731Z env: 2024-08-20T21:44:34.1844979Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:34.1845314Z ##[endgroup] 2024-08-20T21:44:34.1906932Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T21:44:34.1907669Z if systemctl is-active --quiet docker; then 2024-08-20T21:44:34.1908188Z  echo "Docker daemon is running..."; 2024-08-20T21:44:34.1908607Z else 2024-08-20T21:44:34.1909086Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T21:44:34.1909658Z fi 2024-08-20T21:44:34.1914906Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:34.1915367Z env: 2024-08-20T21:44:34.1915622Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:34.1915945Z ##[endgroup] 2024-08-20T21:44:34.1984565Z Docker daemon is running... 2024-08-20T21:44:34.2027037Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T21:44:34.2027552Z with: 2024-08-20T21:44:34.2027770Z shell: bash 2024-08-20T21:44:34.2028030Z timeout_minutes: 5 2024-08-20T21:44:34.2028314Z max_attempts: 3 2024-08-20T21:44:34.2028595Z retry_wait_seconds: 30 2024-08-20T21:44:34.2031657Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T21:44:34.2034704Z polling_interval_seconds: 1 2024-08-20T21:44:34.2035051Z warning_on_retry: true 2024-08-20T21:44:34.2035364Z continue_on_error: false 2024-08-20T21:44:34.2035647Z env: 2024-08-20T21:44:34.2035878Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:34.2036191Z AWS_RETRY_MODE: standard 2024-08-20T21:44:34.2036498Z AWS_MAX_ATTEMPTS: 5 2024-08-20T21:44:34.2036787Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T21:44:34.2037122Z ##[endgroup] 2024-08-20T21:44:35.2666910Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:44:35.2667746Z Configure a credential helper to remove this warning. See 2024-08-20T21:44:35.2668716Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:44:35.2669272Z 2024-08-20T21:44:35.2669375Z Login Succeeded 2024-08-20T21:44:36.2555114Z Command completed after 1 attempt(s). 2024-08-20T21:44:36.2615178Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:44:36.2615922Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:44:36.2616589Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:44:36.2624159Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:36.2624640Z env: 2024-08-20T21:44:36.2624902Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:36.2625233Z ##[endgroup] 2024-08-20T21:44:36.2710463Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T21:44:36.2711234Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T21:44:36.2711819Z # shellcheck disable=SC2046 2024-08-20T21:44:36.2712267Z docker stop $(docker ps -q) || true 2024-08-20T21:44:36.2712717Z # Prune all of the docker images 2024-08-20T21:44:36.2713200Z docker system prune -af 2024-08-20T21:44:36.2752485Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:36.2752960Z env: 2024-08-20T21:44:36.2753198Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:36.2753523Z ##[endgroup] 2024-08-20T21:44:36.3037941Z "docker stop" requires at least 1 argument. 2024-08-20T21:44:36.3038853Z See 'docker stop --help'. 2024-08-20T21:44:36.3039090Z 2024-08-20T21:44:36.3039307Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T21:44:36.3039696Z 2024-08-20T21:44:36.3039832Z Stop one or more running containers 2024-08-20T21:44:36.3186187Z Total reclaimed space: 0B 2024-08-20T21:44:36.3242351Z ##[group]Run set +e 2024-08-20T21:44:36.3242715Z set +e 2024-08-20T21:44:36.3242985Z set -x 2024-08-20T21:44:36.3243264Z  2024-08-20T21:44:36.3243569Z PT_DOMAIN=download.pytorch.org 2024-08-20T21:44:36.3244356Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T21:44:36.3245462Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T21:44:36.3246227Z # one is returned at random 2024-08-20T21:44:36.3246757Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T21:44:36.3247260Z  2024-08-20T21:44:36.3247541Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T21:44:36.3248156Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T21:44:36.3249080Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T21:44:36.3249636Z  2024-08-20T21:44:36.3249927Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T21:44:36.3250467Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T21:44:36.3250967Z  exit 1 2024-08-20T21:44:36.3251263Z  fi 2024-08-20T21:44:36.3251608Z fi 2024-08-20T21:44:36.3251861Z  2024-08-20T21:44:36.3252201Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T21:44:36.3252694Z  # Clean up any old records first 2024-08-20T21:44:36.3253195Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T21:44:36.3253745Z fi 2024-08-20T21:44:36.3253997Z  2024-08-20T21:44:36.3254384Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T21:44:36.3254909Z cat /etc/hosts 2024-08-20T21:44:36.3260563Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:36.3261043Z env: 2024-08-20T21:44:36.3261309Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:36.3261748Z ##[endgroup] 2024-08-20T21:44:36.3284655Z + PT_DOMAIN=download.pytorch.org 2024-08-20T21:44:36.3289711Z ++ dig -4 +short download.pytorch.org 2024-08-20T21:44:36.3290413Z ++ tail -n1 2024-08-20T21:44:36.3622939Z + RESOLVED_IP=108.138.64.125 2024-08-20T21:44:36.3623474Z + '[' -z 108.138.64.125 ']' 2024-08-20T21:44:36.3623988Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T21:44:36.3634418Z 108.138.64.61 download.pytorch.org 2024-08-20T21:44:36.3635962Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-20T21:44:36.6149572Z + echo '108.138.64.125 download.pytorch.org' 2024-08-20T21:44:36.6150123Z + sudo tee -a /etc/hosts 2024-08-20T21:44:36.6531231Z 108.138.64.125 download.pytorch.org 2024-08-20T21:44:36.6546338Z + cat /etc/hosts 2024-08-20T21:44:36.6553246Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T21:44:36.6565281Z ::1 localhost6 localhost6.localdomain6 2024-08-20T21:44:36.6565924Z 108.138.64.125 download.pytorch.org 2024-08-20T21:44:36.6721627Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T21:44:36.6722276Z with: 2024-08-20T21:44:36.6723317Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:36.6724531Z docker-build-dir: .ci/docker 2024-08-20T21:44:36.6724895Z working-directory: . 2024-08-20T21:44:36.6725352Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:36.6725864Z force-push: false 2024-08-20T21:44:36.6726150Z env: 2024-08-20T21:44:36.6726401Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:36.6726726Z ##[endgroup] 2024-08-20T21:44:36.6743664Z ##[group]Run set -ex 2024-08-20T21:44:36.6743993Z set -ex 2024-08-20T21:44:36.6744236Z  2024-08-20T21:44:36.6744744Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T21:44:36.6745716Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T21:44:36.6746458Z # job could then download the pre-built image as usual 2024-08-20T21:44:36.6747150Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T21:44:36.6747784Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:36.6748366Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:36.6748879Z  2024-08-20T21:44:36.6749331Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T21:44:36.6749912Z  exit 0 2024-08-20T21:44:36.6750178Z else 2024-08-20T21:44:36.6750501Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:36.6750896Z fi 2024-08-20T21:44:36.6751132Z  2024-08-20T21:44:36.6751552Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T21:44:36.6752353Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T21:44:36.6753064Z  # use it as it is, but first let's extract the tag 2024-08-20T21:44:36.6753711Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T21:44:36.6754500Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:36.6755147Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:36.6755652Z else 2024-08-20T21:44:36.6756040Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T21:44:36.6756653Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:36.6757491Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:36.6758187Z fi 2024-08-20T21:44:36.6765855Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:36.6766389Z env: 2024-08-20T21:44:36.6766647Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:36.6766977Z REPO_NAME: pytorch 2024-08-20T21:44:36.6768069Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:36.6769250Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T21:44:36.6769739Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:36.6770249Z ##[endgroup] 2024-08-20T21:44:36.6793829Z + [[ ! -d .ci/docker ]] 2024-08-20T21:44:36.6794265Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T21:44:36.6794643Z + echo skip=false 2024-08-20T21:44:36.6796423Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T21:44:36.6801527Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:36.6802705Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T21:44:36.6837397Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:36.6838246Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:36.6839837Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:36.6865498Z ##[group]Run set +e 2024-08-20T21:44:36.6865857Z set +e 2024-08-20T21:44:36.6866140Z set -x 2024-08-20T21:44:36.6866399Z  2024-08-20T21:44:36.6866656Z login() { 2024-08-20T21:44:36.6867317Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T21:44:36.6868039Z } 2024-08-20T21:44:36.6868288Z  2024-08-20T21:44:36.6868542Z retry () { 2024-08-20T21:44:36.6868911Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T21:44:36.6869343Z } 2024-08-20T21:44:36.6869583Z  2024-08-20T21:44:36.6869877Z retry login "${DOCKER_REGISTRY}" 2024-08-20T21:44:36.6870286Z  2024-08-20T21:44:36.6870731Z # Check if image already exists, if it does then skip building it 2024-08-20T21:44:36.6871437Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T21:44:36.6871936Z  exit 0 2024-08-20T21:44:36.6872215Z fi 2024-08-20T21:44:36.6872457Z  2024-08-20T21:44:36.6872944Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T21:44:36.6873781Z # be empty. The default action would be to continue rebuild the image 2024-08-20T21:44:36.6874521Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T21:44:36.6875183Z  # if we're on the base branch then use the parent commit 2024-08-20T21:44:36.6875766Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T21:44:36.6876289Z else 2024-08-20T21:44:36.6876706Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T21:44:36.6877447Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T21:44:36.6877911Z fi 2024-08-20T21:44:36.6878150Z  2024-08-20T21:44:36.6878428Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T21:44:36.6878874Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:36.6879289Z  2024-08-20T21:44:36.6879892Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T21:44:36.6880614Z  exit 0 2024-08-20T21:44:36.6880937Z fi 2024-08-20T21:44:36.6881343Z  2024-08-20T21:44:36.6881767Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T21:44:36.6882757Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T21:44:36.6883561Z  exit 1 2024-08-20T21:44:36.6883841Z fi 2024-08-20T21:44:36.6884096Z  2024-08-20T21:44:36.6884570Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T21:44:36.6885520Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T21:44:36.6886395Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T21:44:36.6887382Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T21:44:36.6888500Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T21:44:36.6889148Z fi 2024-08-20T21:44:36.6889499Z  2024-08-20T21:44:36.6889841Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:36.6895237Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:36.6895719Z env: 2024-08-20T21:44:36.6895959Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:36.6896315Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T21:44:36.6896757Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:44:36.6897959Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:36.6899128Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:36.6899717Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:36.6900218Z ##[endgroup] 2024-08-20T21:44:36.6921694Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:36.6922371Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:36.6923750Z + aws ecr get-login-password --region us-east-1 2024-08-20T21:44:36.6924883Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:37.1982872Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:44:37.1983733Z Configure a credential helper to remove this warning. See 2024-08-20T21:44:37.1984595Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:44:37.1985126Z 2024-08-20T21:44:37.1985235Z Login Succeeded 2024-08-20T21:44:37.1998075Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:37.4038472Z { 2024-08-20T21:44:37.4038980Z "schemaVersion": 2, 2024-08-20T21:44:37.4039562Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T21:44:37.4040186Z "config": { 2024-08-20T21:44:37.4040647Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T21:44:37.4041322Z "size": 43916, 2024-08-20T21:44:37.4041874Z "digest": "sha256:822356886a68936cce34917f5ffc60d864918fab7594bb08c26bf1816dd6110d" 2024-08-20T21:44:37.4042502Z }, 2024-08-20T21:44:37.4042738Z "layers": [ 2024-08-20T21:44:37.4043272Z { 2024-08-20T21:44:37.4043717Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4044274Z "size": 30440714, 2024-08-20T21:44:37.4044849Z "digest": "sha256:762bedf4b1b784c3de6c5022c5307d63123d3b7cdd59211317e37e9d477deaa0" 2024-08-20T21:44:37.4045495Z }, 2024-08-20T21:44:37.4045733Z { 2024-08-20T21:44:37.4046152Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4046731Z "size": 1822, 2024-08-20T21:44:37.4047287Z "digest": "sha256:347beb78d39bc9079a3c20e891908ad5695cdea4a312b0e1de586110bb9ee72f" 2024-08-20T21:44:37.4047923Z }, 2024-08-20T21:44:37.4048142Z { 2024-08-20T21:44:37.4048593Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4049165Z "size": 313164873, 2024-08-20T21:44:37.4049738Z "digest": "sha256:71b614615bcb09bc43ad35a0238b1b82945122c70d95efa9728a6a044a7ddb73" 2024-08-20T21:44:37.4050357Z }, 2024-08-20T21:44:37.4050596Z { 2024-08-20T21:44:37.4051045Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4051608Z "size": 803, 2024-08-20T21:44:37.4052148Z "digest": "sha256:f96c8558662d14e2cc064f94a0166dbf827cf4b783c8d3edb4dbc19dcfff8ca3" 2024-08-20T21:44:37.4052786Z }, 2024-08-20T21:44:37.4053024Z { 2024-08-20T21:44:37.4053444Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4054006Z "size": 106, 2024-08-20T21:44:37.4054557Z "digest": "sha256:c81ff9d20d36fbd48a567918c08f020d23bd83515aec7d23fda14cd56786be81" 2024-08-20T21:44:37.4055904Z + exit 0 2024-08-20T21:44:37.4056175Z }, 2024-08-20T21:44:37.4056421Z { 2024-08-20T21:44:37.4057000Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4057620Z "size": 544, 2024-08-20T21:44:37.4058174Z "digest": "sha256:151e9a9d289467dccd8eefb461af7188b39953d6ac10b678133551bd20586ce6" 2024-08-20T21:44:37.4058813Z }, 2024-08-20T21:44:37.4059038Z { 2024-08-20T21:44:37.4059487Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4060052Z "size": 1214, 2024-08-20T21:44:37.4060627Z "digest": "sha256:8538dcd89f0cac92de23cde0e22b0e7cfde631fbef3062f6387bbc78c4b8c051" 2024-08-20T21:44:37.4061260Z }, 2024-08-20T21:44:37.4061501Z { 2024-08-20T21:44:37.4061940Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4062484Z "size": 483, 2024-08-20T21:44:37.4063087Z "digest": "sha256:cd13007f6f0922ba40ddf0f45d8693c116a5f27e894bd04a34e40eb57bd5c640" 2024-08-20T21:44:37.4063728Z }, 2024-08-20T21:44:37.4064214Z { 2024-08-20T21:44:37.4064647Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4065216Z "size": 110321925, 2024-08-20T21:44:37.4065792Z "digest": "sha256:74fd96b6ebd997dccc9767b9222e3138502d240bca371c2dab11ed193140d9cb" 2024-08-20T21:44:37.4066431Z }, 2024-08-20T21:44:37.4066658Z { 2024-08-20T21:44:37.4067094Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4067665Z "size": 3840, 2024-08-20T21:44:37.4068239Z "digest": "sha256:ce4e1800db98e945fe1b38aac9ac5afcc6edcf1c9cc321499bcb108d04fe1442" 2024-08-20T21:44:37.4068874Z }, 2024-08-20T21:44:37.4069110Z { 2024-08-20T21:44:37.4069544Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4070091Z "size": 1910, 2024-08-20T21:44:37.4070627Z "digest": "sha256:402b067f30bf9e459448062a2a55e42169b09275d1fa427073a332adc6929112" 2024-08-20T21:44:37.4071241Z }, 2024-08-20T21:44:37.4071477Z { 2024-08-20T21:44:37.4071891Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4072457Z "size": 701, 2024-08-20T21:44:37.4073022Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:37.4073670Z }, 2024-08-20T21:44:37.4073890Z { 2024-08-20T21:44:37.4074327Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4075040Z "size": 2663407704, 2024-08-20T21:44:37.4075613Z "digest": "sha256:c6e13ff8aa9a65d1d347f8a336df5ca7febab9d34a3027c2b3c138c1a711baf4" 2024-08-20T21:44:37.4076266Z }, 2024-08-20T21:44:37.4076510Z { 2024-08-20T21:44:37.4076946Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4077492Z "size": 32, 2024-08-20T21:44:37.4078050Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4078690Z }, 2024-08-20T21:44:37.4078932Z { 2024-08-20T21:44:37.4079348Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4079913Z "size": 379, 2024-08-20T21:44:37.4080483Z "digest": "sha256:e238df05ddcc0ec1e1ce769b25f7103f76af6aa35e4aa4ccd1adcdb7621158f6" 2024-08-20T21:44:37.4081261Z }, 2024-08-20T21:44:37.4081486Z { 2024-08-20T21:44:37.4081952Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4082522Z "size": 222887, 2024-08-20T21:44:37.4083059Z "digest": "sha256:a88675bdc72381c487323d232044cca10dd1f77e62eaf00a650bd28d27d6f9bf" 2024-08-20T21:44:37.4083688Z }, 2024-08-20T21:44:37.4083928Z { 2024-08-20T21:44:37.4084362Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4084908Z "size": 231, 2024-08-20T21:44:37.4085451Z "digest": "sha256:f4cb80c411d1b5fb90688374327599a7de6ea89c800473afb1b9766003d5a39a" 2024-08-20T21:44:37.4086080Z }, 2024-08-20T21:44:37.4086320Z { 2024-08-20T21:44:37.4086739Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4087305Z "size": 2933275, 2024-08-20T21:44:37.4087999Z "digest": "sha256:c9589ebd9ae736462b4dc9ea4763b2892eae668c19ca321e7c9ef2762b86a528" 2024-08-20T21:44:37.4088625Z }, 2024-08-20T21:44:37.4088868Z { 2024-08-20T21:44:37.4089305Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4089873Z "size": 1805, 2024-08-20T21:44:37.4090412Z "digest": "sha256:dd51c681db1409845c172ac4fc813dfc9db27bc797fe919a47724410acd481c4" 2024-08-20T21:44:37.4091054Z }, 2024-08-20T21:44:37.4091300Z { 2024-08-20T21:44:37.4091732Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4092277Z "size": 105, 2024-08-20T21:44:37.4092825Z "digest": "sha256:fc983e33a069ae2891a2161f9d24540f9e2728dddc9fd0e63c5b82454ab51f61" 2024-08-20T21:44:37.4093452Z }, 2024-08-20T21:44:37.4093690Z { 2024-08-20T21:44:37.4094103Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4094666Z "size": 564, 2024-08-20T21:44:37.4095217Z "digest": "sha256:bc705d688c776732ddd22c3540b0e135d7dc23e4d46b64918be11b2fa1a265c5" 2024-08-20T21:44:37.4095825Z }, 2024-08-20T21:44:37.4096064Z { 2024-08-20T21:44:37.4096495Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4097058Z "size": 32, 2024-08-20T21:44:37.4097589Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4098232Z }, 2024-08-20T21:44:37.4098471Z { 2024-08-20T21:44:37.4098900Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4099448Z "size": 104, 2024-08-20T21:44:37.4099982Z "digest": "sha256:99f6c1ac758215178233da5104ddb298caf3ae441129209a625e9e649463b980" 2024-08-20T21:44:37.4100600Z }, 2024-08-20T21:44:37.4100821Z { 2024-08-20T21:44:37.4101250Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4101821Z "size": 504, 2024-08-20T21:44:37.4102382Z "digest": "sha256:c409d27a5a9560bf329dec2ed58b701a43f1ffa8babc7ca13c61321cf22ef007" 2024-08-20T21:44:37.4103007Z }, 2024-08-20T21:44:37.4103248Z { 2024-08-20T21:44:37.4103678Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4104245Z "size": 101109909, 2024-08-20T21:44:37.4104813Z "digest": "sha256:a3a398ca1120cf9319dbbb81b12ddc41f7bf3b5bd3f3612c17c2bf7b5194e505" 2024-08-20T21:44:37.4105555Z }, 2024-08-20T21:44:37.4105798Z { 2024-08-20T21:44:37.4106219Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4106790Z "size": 109, 2024-08-20T21:44:37.4107337Z "digest": "sha256:314875f5fae452a321bd50b8527ebb8cf7db0a53ff30293472014390479fcf64" 2024-08-20T21:44:37.4107964Z }, 2024-08-20T21:44:37.4108184Z { 2024-08-20T21:44:37.4108618Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4109180Z "size": 490, 2024-08-20T21:44:37.4109737Z "digest": "sha256:8a5e90dafefa97ef114ac2d4c2f8e000536425318010f40d15bfdc6f8d7e2a79" 2024-08-20T21:44:37.4110360Z }, 2024-08-20T21:44:37.4110598Z { 2024-08-20T21:44:37.4111037Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4111598Z "size": 333, 2024-08-20T21:44:37.4112134Z "digest": "sha256:e3b946ab3378ddc5e96c6355bd0aaa135f29b1ea34b417652442c9df42b6eb49" 2024-08-20T21:44:37.4112776Z }, 2024-08-20T21:44:37.4113021Z { 2024-08-20T21:44:37.4113439Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4114001Z "size": 103, 2024-08-20T21:44:37.4114528Z "digest": "sha256:14a85a990f7189af056863e625917e6a5583ce8a7b1d319783011803ea1a9da4" 2024-08-20T21:44:37.4115144Z }, 2024-08-20T21:44:37.4115365Z { 2024-08-20T21:44:37.4115794Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4116356Z "size": 1473, 2024-08-20T21:44:37.4116893Z "digest": "sha256:16d809b167d49638350b86af396a05eca356ba903ca4953a083d29f28154c16a" 2024-08-20T21:44:37.4117495Z }, 2024-08-20T21:44:37.4117737Z { 2024-08-20T21:44:37.4118250Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4118822Z "size": 611525039, 2024-08-20T21:44:37.4119380Z "digest": "sha256:9b1a95eedb8338e08a9ba00b933ef64420d41664b1a4c2a40f1703c79a3d2ba1" 2024-08-20T21:44:37.4120015Z }, 2024-08-20T21:44:37.4120258Z { 2024-08-20T21:44:37.4120679Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4121379Z "size": 164, 2024-08-20T21:44:37.4121923Z "digest": "sha256:b35ca05427236370ed98f905cf9b8a0e4599173ef1843504fc2547e9525cd483" 2024-08-20T21:44:37.4122551Z }, 2024-08-20T21:44:37.4122773Z { 2024-08-20T21:44:37.4123209Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4123771Z "size": 2210, 2024-08-20T21:44:37.4124333Z "digest": "sha256:bad10ac3a6279679db06f845b321a9fdaef80be969acbfd257f9e807762953a8" 2024-08-20T21:44:37.4124955Z }, 2024-08-20T21:44:37.4125194Z { 2024-08-20T21:44:37.4125626Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4126179Z "size": 152, 2024-08-20T21:44:37.4126739Z "digest": "sha256:943da2164c7a7bf690bcbc751afe5ea646925ddf0ab96fd2ce48b0a52bd66e9d" 2024-08-20T21:44:37.4127379Z }, 2024-08-20T21:44:37.4127620Z { 2024-08-20T21:44:37.4128036Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4128606Z "size": 648, 2024-08-20T21:44:37.4129149Z "digest": "sha256:2d276441bd8ae528772f4fd876af429ab3518bebf61090795ad3b6aced9d60e4" 2024-08-20T21:44:37.4129784Z }, 2024-08-20T21:44:37.4130006Z { 2024-08-20T21:44:37.4130438Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4130999Z "size": 32, 2024-08-20T21:44:37.4131549Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4132165Z }, 2024-08-20T21:44:37.4132405Z { 2024-08-20T21:44:37.4132836Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4133380Z "size": 167, 2024-08-20T21:44:37.4133934Z "digest": "sha256:a3ef8d7d02be75588561825a22ecbdc3129ad8a543f4d1415fee8284d91327bb" 2024-08-20T21:44:37.4134570Z }, 2024-08-20T21:44:37.4134808Z { 2024-08-20T21:44:37.4135223Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4135887Z "size": 109, 2024-08-20T21:44:37.4136438Z "digest": "sha256:6aaed7a7874900b7539103d4ef762ab0f9a2d760c898f2ebcf135b73858aeed5" 2024-08-20T21:44:37.4137076Z }, 2024-08-20T21:44:37.4137300Z { 2024-08-20T21:44:37.4137738Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4138308Z "size": 102, 2024-08-20T21:44:37.4138831Z "digest": "sha256:944e780839169e63df50b5cc05cc9fb99e8ed0b9692d56cee9cd06137bdb08e3" 2024-08-20T21:44:37.4139459Z }, 2024-08-20T21:44:37.4139698Z { 2024-08-20T21:44:37.4140132Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4140677Z "size": 422, 2024-08-20T21:44:37.4141224Z "digest": "sha256:9602de36240c11e5eb9a1926c6443a2ff43948a958a15162e7aa827e1f83ad3e" 2024-08-20T21:44:37.4141846Z }, 2024-08-20T21:44:37.4142084Z { 2024-08-20T21:44:37.4142635Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4143204Z "size": 32, 2024-08-20T21:44:37.4143761Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4144405Z }, 2024-08-20T21:44:37.4144628Z { 2024-08-20T21:44:37.4145067Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4145630Z "size": 111, 2024-08-20T21:44:37.4146154Z "digest": "sha256:6d3ac49638ec41e1302ba5b1028a55629b0bb2cb2603212aba86d0058783ab50" 2024-08-20T21:44:37.4146781Z }, 2024-08-20T21:44:37.4147026Z { 2024-08-20T21:44:37.4147457Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4148004Z "size": 473, 2024-08-20T21:44:37.4148687Z "digest": "sha256:c5c5dbfb1dcc6e7ef2cabd2a583442e21c5b95a0cb625f0766592fff0b807d95" 2024-08-20T21:44:37.4149346Z }, 2024-08-20T21:44:37.4149589Z { 2024-08-20T21:44:37.4150009Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4150576Z "size": 32, 2024-08-20T21:44:37.4151129Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4151776Z }, 2024-08-20T21:44:37.4152003Z { 2024-08-20T21:44:37.4152440Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4153005Z "size": 112, 2024-08-20T21:44:37.4153543Z "digest": "sha256:327bd4a08d3c4e9ae265be2b8ce0d8ce4e4016f60e4c02a36c638789adf49ea5" 2024-08-20T21:44:37.4154184Z }, 2024-08-20T21:44:37.4154425Z { 2024-08-20T21:44:37.4154857Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4155399Z "size": 566, 2024-08-20T21:44:37.4155934Z "digest": "sha256:6c520cbfb1595425e6e07ad7e2952222365e91d02cf25a4ec1f8565279b941de" 2024-08-20T21:44:37.4156562Z }, 2024-08-20T21:44:37.4156800Z { 2024-08-20T21:44:37.4157211Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4157771Z "size": 43163722, 2024-08-20T21:44:37.4158332Z "digest": "sha256:a8cda4c989262a43930b0bf2507986800779f6a6ec65f8ef64bbfa72c5ca5bf6" 2024-08-20T21:44:37.4158945Z }, 2024-08-20T21:44:37.4159353Z { 2024-08-20T21:44:37.4159782Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4160343Z "size": 106, 2024-08-20T21:44:37.4160967Z "digest": "sha256:3d995c530b49c1eecd803a2cfde69821f88cc0b8b5429d8955a65319217337d6" 2024-08-20T21:44:37.4161608Z }, 2024-08-20T21:44:37.4161848Z { 2024-08-20T21:44:37.4162371Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4162922Z "size": 295, 2024-08-20T21:44:37.4163455Z "digest": "sha256:51635df36934a1409214ed1d1c03c997d8597887f34e547bca5fb5373e509ca8" 2024-08-20T21:44:37.4164252Z }, 2024-08-20T21:44:37.4164491Z { 2024-08-20T21:44:37.4164914Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4165522Z "size": 32, 2024-08-20T21:44:37.4166070Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4166687Z }, 2024-08-20T21:44:37.4167058Z { 2024-08-20T21:44:37.4167494Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4168059Z "size": 106, 2024-08-20T21:44:37.4168591Z "digest": "sha256:58f9d093cf6898c8d0fe441806e8afb6fcf8180d6ce0e61f3f8e1a5d27872b95" 2024-08-20T21:44:37.4169231Z }, 2024-08-20T21:44:37.4169471Z { 2024-08-20T21:44:37.4169906Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4170455Z "size": 425, 2024-08-20T21:44:37.4171024Z "digest": "sha256:db291b17fdf0ea8442b4ee4ccf970b6bcbbf230840366e5dfe62faf43175568d" 2024-08-20T21:44:37.4171672Z }, 2024-08-20T21:44:37.4171895Z { 2024-08-20T21:44:37.4172333Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4172900Z "size": 19308700, 2024-08-20T21:44:37.4173459Z "digest": "sha256:4390932d47dc99aad011fc417cb24b8cd895f84e0247e7b1930a974848f4ae49" 2024-08-20T21:44:37.4174070Z }, 2024-08-20T21:44:37.4174311Z { 2024-08-20T21:44:37.4174747Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4175316Z "size": 108, 2024-08-20T21:44:37.4175959Z "digest": "sha256:d959da1ed4a86d0902fc218b87883ea901725726db56d746621bfeb206d18ad6" 2024-08-20T21:44:37.4176543Z }, 2024-08-20T21:44:37.4176772Z { 2024-08-20T21:44:37.4177172Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4177864Z "size": 438, 2024-08-20T21:44:37.4178495Z "digest": "sha256:82769435ccd5317b20def22a2e4295f283bac4ed179599487713032c3609f189" 2024-08-20T21:44:37.4179074Z }, 2024-08-20T21:44:37.4179277Z { 2024-08-20T21:44:37.4179769Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4180300Z "size": 701, 2024-08-20T21:44:37.4180829Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:37.4181414Z }, 2024-08-20T21:44:37.4181641Z { 2024-08-20T21:44:37.4182043Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4182567Z "size": 141, 2024-08-20T21:44:37.4183052Z "digest": "sha256:50078e7792d3b05b7c50ff90f27eb679dee945052c25d2fa8590cbd90dee9ac0" 2024-08-20T21:44:37.4183633Z }, 2024-08-20T21:44:37.4183854Z { 2024-08-20T21:44:37.4184239Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4184759Z "size": 135, 2024-08-20T21:44:37.4185250Z "digest": "sha256:557fee3a8871f019075f948770fb5f0860218079c7dd47fcc7982499991fb2a1" 2024-08-20T21:44:37.4185818Z }, 2024-08-20T21:44:37.4186022Z { 2024-08-20T21:44:37.4186423Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4187134Z "size": 4841634229, 2024-08-20T21:44:37.4187761Z "digest": "sha256:cc0b3fc4f447d5036748372379999ebaac2ba414c77302c2910751054210e216" 2024-08-20T21:44:37.4188346Z }, 2024-08-20T21:44:37.4188577Z { 2024-08-20T21:44:37.4189000Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4189738Z "size": 194, 2024-08-20T21:44:37.4190254Z "digest": "sha256:f18a0119669a20a0f6987e879f10f84bd03b89b7865214f4904cd1b9987d0b0e" 2024-08-20T21:44:37.4190861Z }, 2024-08-20T21:44:37.4191096Z { 2024-08-20T21:44:37.4191506Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4192064Z "size": 1379, 2024-08-20T21:44:37.4192613Z "digest": "sha256:6197a6a036aca1f62c286b2b81941a497fa5ffed336bca2675084fe69f48f405" 2024-08-20T21:44:37.4193247Z }, 2024-08-20T21:44:37.4193465Z { 2024-08-20T21:44:37.4193896Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4194453Z "size": 701, 2024-08-20T21:44:37.4195022Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:37.4195644Z }, 2024-08-20T21:44:37.4195885Z { 2024-08-20T21:44:37.4196316Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4196876Z "size": 139, 2024-08-20T21:44:37.4197497Z "digest": "sha256:ebdeb237156e5f92ea7a5aa15d85a86b5c64ae0e6ca749609008f2e6eee128b8" 2024-08-20T21:44:37.4198144Z }, 2024-08-20T21:44:37.4198386Z { 2024-08-20T21:44:37.4198803Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4199486Z "size": 3733332465, 2024-08-20T21:44:37.4200054Z "digest": "sha256:fd5d0d6ba583ac4b9de591cc4925403acd2588d10fc04114b681beb022597580" 2024-08-20T21:44:37.4200675Z }, 2024-08-20T21:44:37.4200986Z { 2024-08-20T21:44:37.4201602Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4202215Z "size": 157, 2024-08-20T21:44:37.4202770Z "digest": "sha256:6b068755e588d5e5994af07fe71bd40adbbc544f6238dea5511e69862a5236dd" 2024-08-20T21:44:37.4203381Z }, 2024-08-20T21:44:37.4203620Z { 2024-08-20T21:44:37.4204053Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4204597Z "size": 849, 2024-08-20T21:44:37.4205133Z "digest": "sha256:371497d14c4e377a762eec74e0c7575051758f8aa74cdc2f2436fb382df849d3" 2024-08-20T21:44:37.4205758Z }, 2024-08-20T21:44:37.4205995Z { 2024-08-20T21:44:37.4206412Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4206973Z "size": 701, 2024-08-20T21:44:37.4207535Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:37.4208182Z }, 2024-08-20T21:44:37.4208399Z { 2024-08-20T21:44:37.4208826Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4209387Z "size": 139, 2024-08-20T21:44:37.4209924Z "digest": "sha256:4b50cb25a6ac9133e2c7894845496771a2f9b23eb35a58a778bb989f9bfbac5e" 2024-08-20T21:44:37.4210608Z }, 2024-08-20T21:44:37.4210845Z { 2024-08-20T21:44:37.4211276Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4211821Z "size": 32, 2024-08-20T21:44:37.4212375Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4213021Z }, 2024-08-20T21:44:37.4213263Z { 2024-08-20T21:44:37.4213780Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4214305Z "size": 159, 2024-08-20T21:44:37.4214813Z "digest": "sha256:889d8139c009597aca470a64bf7aaea6d509fd8ee9a96187fa784a1d9931f350" 2024-08-20T21:44:37.4215399Z }, 2024-08-20T21:44:37.4215610Z { 2024-08-20T21:44:37.4216015Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4216543Z "size": 907, 2024-08-20T21:44:37.4217050Z "digest": "sha256:2bcabd118f67bf8b2b8c6eb97e245fef2c7753e07e4ecf062b3cf6c905bdea81" 2024-08-20T21:44:37.4217652Z }, 2024-08-20T21:44:37.4217884Z { 2024-08-20T21:44:37.4218287Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4218791Z "size": 701, 2024-08-20T21:44:37.4219314Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:37.4219919Z }, 2024-08-20T21:44:37.4220144Z { 2024-08-20T21:44:37.4220528Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4221052Z "size": 134, 2024-08-20T21:44:37.4221569Z "digest": "sha256:1d831c1ea7c74b810f149c31d6eebc2d2bed6a5fb9179a9b230fec1c85f1094e" 2024-08-20T21:44:37.4222166Z }, 2024-08-20T21:44:37.4222371Z { 2024-08-20T21:44:37.4222777Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4223298Z "size": 32, 2024-08-20T21:44:37.4223792Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4224382Z }, 2024-08-20T21:44:37.4239842Z { 2024-08-20T21:44:37.4240426Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4241083Z "size": 158, 2024-08-20T21:44:37.4241808Z "digest": "sha256:d29d428148005386a1f6f19dd2590d9cbf58a6c1f94898e1e6ac92ea666f1641" 2024-08-20T21:44:37.4242421Z }, 2024-08-20T21:44:37.4242661Z { 2024-08-20T21:44:37.4243243Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4243808Z "size": 1579, 2024-08-20T21:44:37.4244338Z "digest": "sha256:10df3ed01753e29d9ea7d799510d1d4534671a9a4ac53db74998c1ba9c38a48c" 2024-08-20T21:44:37.4244970Z }, 2024-08-20T21:44:37.4245207Z { 2024-08-20T21:44:37.4245640Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4246183Z "size": 32, 2024-08-20T21:44:37.4246727Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4247359Z }, 2024-08-20T21:44:37.4247576Z { 2024-08-20T21:44:37.4248010Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4248574Z "size": 136, 2024-08-20T21:44:37.4249101Z "digest": "sha256:3b3541d6658c782e6cf71ee8b9268b351f32109e8571447dc8993e649742d905" 2024-08-20T21:44:37.4249702Z }, 2024-08-20T21:44:37.4249939Z { 2024-08-20T21:44:37.4250362Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4250932Z "size": 378, 2024-08-20T21:44:37.4251447Z "digest": "sha256:5e997601a6320f613b759c5684ae83fa39646d3c85a349797faba2e4fbc70d83" 2024-08-20T21:44:37.4252061Z }, 2024-08-20T21:44:37.4252297Z { 2024-08-20T21:44:37.4252727Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4253271Z "size": 32, 2024-08-20T21:44:37.4253914Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4254507Z }, 2024-08-20T21:44:37.4254714Z { 2024-08-20T21:44:37.4255103Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4255724Z "size": 104, 2024-08-20T21:44:37.4256227Z "digest": "sha256:d2924613594f110f7aa5b1681ba6bc1044cd355b5eeb17944cd11531d33a1552" 2024-08-20T21:44:37.4256789Z }, 2024-08-20T21:44:37.4257012Z { 2024-08-20T21:44:37.4257412Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4257932Z "size": 1841, 2024-08-20T21:44:37.4258421Z "digest": "sha256:f66c8294ffd00da25739af287f89a292ee2b06597012e6abac6947e2197961ae" 2024-08-20T21:44:37.4258997Z }, 2024-08-20T21:44:37.4259220Z { 2024-08-20T21:44:37.4259760Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4260277Z "size": 7529594, 2024-08-20T21:44:37.4260805Z "digest": "sha256:9da0ec93b0d7d37059844ca82dd8b0ce33bdd51614cc9bd66c391ffe64094ac7" 2024-08-20T21:44:37.4261396Z }, 2024-08-20T21:44:37.4261600Z { 2024-08-20T21:44:37.4261998Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4262517Z "size": 106, 2024-08-20T21:44:37.4263011Z "digest": "sha256:d476e4b8627568a99c85e93d60a7ce5676a42e9ee10a5b1054348658d77f56f3" 2024-08-20T21:44:37.4263569Z }, 2024-08-20T21:44:37.4263794Z { 2024-08-20T21:44:37.4264550Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4265289Z "size": 163, 2024-08-20T21:44:37.4265892Z "digest": "sha256:397c513e7e07f5c83f0d0c8c407a7bdc3e7167a57744262cd5a1282ee451d52d" 2024-08-20T21:44:37.4266525Z }, 2024-08-20T21:44:37.4266764Z { 2024-08-20T21:44:37.4267179Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4267744Z "size": 7943, 2024-08-20T21:44:37.4268297Z "digest": "sha256:5d6d58e6432b6e8048d3f6cccb787a1cfe9d1203f742a9f0fddfb00338cccae7" 2024-08-20T21:44:37.4268934Z }, 2024-08-20T21:44:37.4269162Z { 2024-08-20T21:44:37.4269602Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4270173Z "size": 8066, 2024-08-20T21:44:37.4270740Z "digest": "sha256:f9db7741af1464b9e28a0bab667a6b4559ab7f3d319add7a50bc1dcc2b1e97fe" 2024-08-20T21:44:37.4271360Z }, 2024-08-20T21:44:37.4271605Z { 2024-08-20T21:44:37.4272043Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4272594Z "size": 303, 2024-08-20T21:44:37.4273138Z "digest": "sha256:c0f5c95392e3593f2bccab197736341bfef11427975492f581c088b8e21401a1" 2024-08-20T21:44:37.4273918Z }, 2024-08-20T21:44:37.4274156Z { 2024-08-20T21:44:37.4274567Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4275127Z "size": 32, 2024-08-20T21:44:37.4275679Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4276319Z }, 2024-08-20T21:44:37.4276537Z { 2024-08-20T21:44:37.4276965Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4277608Z "size": 108, 2024-08-20T21:44:37.4278115Z "digest": "sha256:d6d4b368f5e67517827a02d0e324ced915843fa69780cbf87cac205ffcf04032" 2024-08-20T21:44:37.4278680Z }, 2024-08-20T21:44:37.4278900Z { 2024-08-20T21:44:37.4279306Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4279808Z "size": 54145664, 2024-08-20T21:44:37.4280333Z "digest": "sha256:6e341d8be4cd61d3439f2becbc390d9d27179704631d29225ba4c01bc518ff56" 2024-08-20T21:44:37.4281002Z }, 2024-08-20T21:44:37.4281406Z { 2024-08-20T21:44:37.4281820Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:37.4282380Z "size": 32, 2024-08-20T21:44:37.4282926Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:37.4283556Z } 2024-08-20T21:44:37.4283772Z ] 2024-08-20T21:44:37.4284004Z } 2024-08-20T21:44:37.4384080Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T21:44:37.4384526Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T21:44:37.4385036Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T21:44:37.4391428Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:37.4391892Z env: 2024-08-20T21:44:37.4392141Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:37.4393172Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:37.4394233Z ##[endgroup] 2024-08-20T21:44:37.4420027Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:37.4471213Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T21:44:37.4471796Z with: 2024-08-20T21:44:37.4472828Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:37.4474107Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:37.4474626Z env: 2024-08-20T21:44:37.4474890Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:37.4475227Z ##[endgroup] 2024-08-20T21:44:37.4491039Z ##[group]Run set -x 2024-08-20T21:44:37.4491366Z set -x 2024-08-20T21:44:37.4491652Z set +e 2024-08-20T21:44:37.4491930Z  2024-08-20T21:44:37.4492171Z login() { 2024-08-20T21:44:37.4492836Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T21:44:37.4493670Z } 2024-08-20T21:44:37.4493911Z  2024-08-20T21:44:37.4494183Z retry () { 2024-08-20T21:44:37.4494534Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T21:44:37.4494924Z } 2024-08-20T21:44:37.4495165Z  2024-08-20T21:44:37.4495442Z retry login "${DOCKER_REGISTRY}" 2024-08-20T21:44:37.4495807Z  2024-08-20T21:44:37.4496051Z set -e 2024-08-20T21:44:37.4496482Z # ignore output since only exit code is used for conditional 2024-08-20T21:44:37.4497134Z # only pull docker image if it's not available locally 2024-08-20T21:44:37.4497836Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T21:44:37.4498487Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T21:44:37.4498886Z fi 2024-08-20T21:44:37.4504076Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:37.4504527Z env: 2024-08-20T21:44:37.4504779Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:37.4505805Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:37.4506984Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:37.4507463Z ##[endgroup] 2024-08-20T21:44:37.4529532Z + set +e 2024-08-20T21:44:37.4530149Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:37.4530811Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:37.4532799Z + aws ecr get-login-password --region us-east-1 2024-08-20T21:44:37.4534029Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:37.9556212Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:44:37.9557094Z Configure a credential helper to remove this warning. See 2024-08-20T21:44:37.9557961Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:44:37.9558492Z 2024-08-20T21:44:37.9558626Z Login Succeeded 2024-08-20T21:44:37.9568596Z + set -e 2024-08-20T21:44:37.9569883Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:37.9696727Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:37.9698832Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:38.2223693Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks 2024-08-20T21:44:38.2233845Z 762bedf4b1b7: Pulling fs layer 2024-08-20T21:44:38.2234498Z 347beb78d39b: Pulling fs layer 2024-08-20T21:44:38.2235218Z 71b614615bcb: Pulling fs layer 2024-08-20T21:44:38.2236171Z f96c8558662d: Pulling fs layer 2024-08-20T21:44:38.2236829Z c81ff9d20d36: Pulling fs layer 2024-08-20T21:44:38.2237467Z 151e9a9d2894: Pulling fs layer 2024-08-20T21:44:38.2238089Z 8538dcd89f0c: Pulling fs layer 2024-08-20T21:44:38.2238732Z cd13007f6f09: Pulling fs layer 2024-08-20T21:44:38.2239149Z 74fd96b6ebd9: Pulling fs layer 2024-08-20T21:44:38.2239526Z ce4e1800db98: Pulling fs layer 2024-08-20T21:44:38.2239881Z 402b067f30bf: Pulling fs layer 2024-08-20T21:44:38.2240263Z 2cebdfe4c5a8: Pulling fs layer 2024-08-20T21:44:38.2240643Z c6e13ff8aa9a: Pulling fs layer 2024-08-20T21:44:38.2241143Z 4f4fb700ef54: Pulling fs layer 2024-08-20T21:44:38.2241503Z e238df05ddcc: Pulling fs layer 2024-08-20T21:44:38.2241880Z a88675bdc723: Pulling fs layer 2024-08-20T21:44:38.2242252Z f4cb80c411d1: Pulling fs layer 2024-08-20T21:44:38.2242713Z c9589ebd9ae7: Pulling fs layer 2024-08-20T21:44:38.2243084Z dd51c681db14: Pulling fs layer 2024-08-20T21:44:38.2243442Z fc983e33a069: Pulling fs layer 2024-08-20T21:44:38.2243822Z bc705d688c77: Pulling fs layer 2024-08-20T21:44:38.2244198Z 99f6c1ac7582: Pulling fs layer 2024-08-20T21:44:38.2244565Z c409d27a5a95: Pulling fs layer 2024-08-20T21:44:38.2244908Z c81ff9d20d36: Waiting 2024-08-20T21:44:38.2245237Z a3a398ca1120: Pulling fs layer 2024-08-20T21:44:38.2245590Z 151e9a9d2894: Waiting 2024-08-20T21:44:38.2245887Z 8538dcd89f0c: Waiting 2024-08-20T21:44:38.2246210Z 314875f5fae4: Pulling fs layer 2024-08-20T21:44:38.2246566Z cd13007f6f09: Waiting 2024-08-20T21:44:38.2246894Z 8a5e90dafefa: Pulling fs layer 2024-08-20T21:44:38.2247256Z e3b946ab3378: Pulling fs layer 2024-08-20T21:44:38.2247633Z 14a85a990f71: Pulling fs layer 2024-08-20T21:44:38.2248000Z 16d809b167d4: Pulling fs layer 2024-08-20T21:44:38.2248338Z 74fd96b6ebd9: Waiting 2024-08-20T21:44:38.2248664Z 9b1a95eedb83: Pulling fs layer 2024-08-20T21:44:38.2249233Z b35ca0542723: Pulling fs layer 2024-08-20T21:44:38.2249590Z ce4e1800db98: Waiting 2024-08-20T21:44:38.2249906Z bad10ac3a627: Pulling fs layer 2024-08-20T21:44:38.2250276Z 402b067f30bf: Waiting 2024-08-20T21:44:38.2250605Z 943da2164c7a: Pulling fs layer 2024-08-20T21:44:38.2250980Z 2d276441bd8a: Pulling fs layer 2024-08-20T21:44:38.2251337Z a3ef8d7d02be: Pulling fs layer 2024-08-20T21:44:38.2251698Z 2cebdfe4c5a8: Waiting 2024-08-20T21:44:38.2252028Z 6aaed7a78749: Pulling fs layer 2024-08-20T21:44:38.2252369Z c6e13ff8aa9a: Waiting 2024-08-20T21:44:38.2252693Z 944e78083916: Pulling fs layer 2024-08-20T21:44:38.2253055Z 9602de36240c: Pulling fs layer 2024-08-20T21:44:38.2253413Z 4f4fb700ef54: Waiting 2024-08-20T21:44:38.2253709Z 9b1a95eedb83: Waiting 2024-08-20T21:44:38.2254017Z f96c8558662d: Waiting 2024-08-20T21:44:38.2254324Z c409d27a5a95: Waiting 2024-08-20T21:44:38.2254634Z b35ca0542723: Waiting 2024-08-20T21:44:38.2254940Z 6d3ac49638ec: Pulling fs layer 2024-08-20T21:44:38.2255299Z a3a398ca1120: Waiting 2024-08-20T21:44:38.2255604Z 314875f5fae4: Waiting 2024-08-20T21:44:38.2255897Z e3b946ab3378: Waiting 2024-08-20T21:44:38.2256224Z c5c5dbfb1dcc: Pulling fs layer 2024-08-20T21:44:38.2256584Z a88675bdc723: Waiting 2024-08-20T21:44:38.2256890Z 14a85a990f71: Waiting 2024-08-20T21:44:38.2257181Z bad10ac3a627: Waiting 2024-08-20T21:44:38.2257493Z f4cb80c411d1: Waiting 2024-08-20T21:44:38.2257800Z 16d809b167d4: Waiting 2024-08-20T21:44:38.2258093Z e238df05ddcc: Waiting 2024-08-20T21:44:38.2258423Z 327bd4a08d3c: Pulling fs layer 2024-08-20T21:44:38.2258777Z 943da2164c7a: Waiting 2024-08-20T21:44:38.2259098Z 6c520cbfb159: Pulling fs layer 2024-08-20T21:44:38.2259438Z c9589ebd9ae7: Waiting 2024-08-20T21:44:38.2259771Z a8cda4c98926: Pulling fs layer 2024-08-20T21:44:38.2260143Z 3d995c530b49: Pulling fs layer 2024-08-20T21:44:38.2260509Z 51635df36934: Pulling fs layer 2024-08-20T21:44:38.2260846Z 8a5e90dafefa: Waiting 2024-08-20T21:44:38.2261160Z 99f6c1ac7582: Waiting 2024-08-20T21:44:38.2261488Z 58f9d093cf68: Pulling fs layer 2024-08-20T21:44:38.2261829Z 6aaed7a78749: Waiting 2024-08-20T21:44:38.2262161Z db291b17fdf0: Pulling fs layer 2024-08-20T21:44:38.2262513Z 9602de36240c: Waiting 2024-08-20T21:44:38.2262963Z 4390932d47dc: Pulling fs layer 2024-08-20T21:44:38.2263315Z 6d3ac49638ec: Waiting 2024-08-20T21:44:38.2263628Z c5c5dbfb1dcc: Waiting 2024-08-20T21:44:38.2264223Z d959da1ed4a8: Pulling fs layer 2024-08-20T21:44:38.2264568Z 2d276441bd8a: Waiting 2024-08-20T21:44:38.2264877Z 944e78083916: Waiting 2024-08-20T21:44:38.2265200Z 82769435ccd5: Pulling fs layer 2024-08-20T21:44:38.2265562Z a3ef8d7d02be: Waiting 2024-08-20T21:44:38.2265862Z 58f9d093cf68: Waiting 2024-08-20T21:44:38.2266175Z db291b17fdf0: Waiting 2024-08-20T21:44:38.2266485Z 327bd4a08d3c: Waiting 2024-08-20T21:44:38.2266814Z a8cda4c98926: Waiting 2024-08-20T21:44:38.2267108Z 3d995c530b49: Waiting 2024-08-20T21:44:38.2267429Z 50078e7792d3: Pulling fs layer 2024-08-20T21:44:38.2267804Z 557fee3a8871: Pulling fs layer 2024-08-20T21:44:38.2268155Z 82769435ccd5: Waiting 2024-08-20T21:44:38.2268459Z 50078e7792d3: Waiting 2024-08-20T21:44:38.2268833Z cc0b3fc4f447: Pulling fs layer 2024-08-20T21:44:38.2269237Z 557fee3a8871: Waiting 2024-08-20T21:44:38.2269580Z 6c520cbfb159: Waiting 2024-08-20T21:44:38.2269935Z f18a0119669a: Pulling fs layer 2024-08-20T21:44:38.2270294Z cc0b3fc4f447: Waiting 2024-08-20T21:44:38.2270648Z 6197a6a036ac: Pulling fs layer 2024-08-20T21:44:38.2271011Z fc983e33a069: Waiting 2024-08-20T21:44:38.2271318Z 4390932d47dc: Waiting 2024-08-20T21:44:38.2271630Z ebdeb237156e: Pulling fs layer 2024-08-20T21:44:38.2272012Z fd5d0d6ba583: Pulling fs layer 2024-08-20T21:44:38.2272374Z f18a0119669a: Waiting 2024-08-20T21:44:38.2272680Z 6b068755e588: Pulling fs layer 2024-08-20T21:44:38.2273040Z bc705d688c77: Waiting 2024-08-20T21:44:38.2273354Z d959da1ed4a8: Waiting 2024-08-20T21:44:38.2273684Z 371497d14c4e: Pulling fs layer 2024-08-20T21:44:38.2274027Z ebdeb237156e: Waiting 2024-08-20T21:44:38.2274343Z 51635df36934: Waiting 2024-08-20T21:44:38.2274816Z 371497d14c4e: Waiting 2024-08-20T21:44:38.2275145Z 4b50cb25a6ac: Pulling fs layer 2024-08-20T21:44:38.2275485Z 6b068755e588: Waiting 2024-08-20T21:44:38.2275793Z 6197a6a036ac: Waiting 2024-08-20T21:44:38.2276120Z 889d8139c009: Pulling fs layer 2024-08-20T21:44:38.2276465Z 4b50cb25a6ac: Waiting 2024-08-20T21:44:38.2276797Z 2bcabd118f67: Pulling fs layer 2024-08-20T21:44:38.2277160Z 2bcabd118f67: Waiting 2024-08-20T21:44:38.2277469Z 889d8139c009: Waiting 2024-08-20T21:44:38.2277779Z 1d831c1ea7c7: Pulling fs layer 2024-08-20T21:44:38.2278148Z d29d42814800: Pulling fs layer 2024-08-20T21:44:38.2278519Z 10df3ed01753: Pulling fs layer 2024-08-20T21:44:38.2278857Z 1d831c1ea7c7: Waiting 2024-08-20T21:44:38.2279184Z 3b3541d6658c: Pulling fs layer 2024-08-20T21:44:38.2279552Z 5e997601a632: Pulling fs layer 2024-08-20T21:44:38.2279906Z 10df3ed01753: Waiting 2024-08-20T21:44:38.2280198Z 3b3541d6658c: Waiting 2024-08-20T21:44:38.2280523Z d2924613594f: Pulling fs layer 2024-08-20T21:44:38.2281047Z f66c8294ffd0: Pulling fs layer 2024-08-20T21:44:38.2281424Z 9da0ec93b0d7: Pulling fs layer 2024-08-20T21:44:38.2281769Z f66c8294ffd0: Waiting 2024-08-20T21:44:38.2282095Z d476e4b86275: Pulling fs layer 2024-08-20T21:44:38.2282458Z 9da0ec93b0d7: Waiting 2024-08-20T21:44:38.2282770Z 397c513e7e07: Pulling fs layer 2024-08-20T21:44:38.2283140Z 5d6d58e6432b: Pulling fs layer 2024-08-20T21:44:38.2283510Z f9db7741af14: Pulling fs layer 2024-08-20T21:44:38.2283868Z 397c513e7e07: Waiting 2024-08-20T21:44:38.2284161Z d476e4b86275: Waiting 2024-08-20T21:44:38.2284485Z c0f5c95392e3: Pulling fs layer 2024-08-20T21:44:38.2284846Z f9db7741af14: Waiting 2024-08-20T21:44:38.2285174Z d6d4b368f5e6: Pulling fs layer 2024-08-20T21:44:38.2285531Z 6e341d8be4cd: Pulling fs layer 2024-08-20T21:44:38.2285890Z 6e341d8be4cd: Waiting 2024-08-20T21:44:38.2286201Z d6d4b368f5e6: Waiting 2024-08-20T21:44:38.2930350Z 347beb78d39b: Download complete 2024-08-20T21:44:38.3761060Z f96c8558662d: Verifying Checksum 2024-08-20T21:44:38.3762297Z f96c8558662d: Download complete 2024-08-20T21:44:38.4536497Z c81ff9d20d36: Verifying Checksum 2024-08-20T21:44:38.4536936Z c81ff9d20d36: Download complete 2024-08-20T21:44:38.5424795Z 151e9a9d2894: Verifying Checksum 2024-08-20T21:44:38.5425487Z 151e9a9d2894: Download complete 2024-08-20T21:44:38.5744497Z 762bedf4b1b7: Download complete 2024-08-20T21:44:38.6149966Z 8538dcd89f0c: Download complete 2024-08-20T21:44:38.6405220Z cd13007f6f09: Download complete 2024-08-20T21:44:38.7139249Z ce4e1800db98: Verifying Checksum 2024-08-20T21:44:38.7139790Z ce4e1800db98: Download complete 2024-08-20T21:44:38.7764356Z 402b067f30bf: Verifying Checksum 2024-08-20T21:44:38.7764833Z 402b067f30bf: Download complete 2024-08-20T21:44:38.8547171Z 2cebdfe4c5a8: Verifying Checksum 2024-08-20T21:44:38.8547979Z 2cebdfe4c5a8: Download complete 2024-08-20T21:44:39.3268214Z 762bedf4b1b7: Pull complete 2024-08-20T21:44:39.3476605Z 347beb78d39b: Pull complete 2024-08-20T21:44:39.7968808Z 74fd96b6ebd9: Verifying Checksum 2024-08-20T21:44:39.7969276Z 74fd96b6ebd9: Download complete 2024-08-20T21:44:39.8069356Z 4f4fb700ef54: Download complete 2024-08-20T21:44:39.8869881Z e238df05ddcc: Download complete 2024-08-20T21:44:39.9726115Z a88675bdc723: Verifying Checksum 2024-08-20T21:44:39.9726574Z a88675bdc723: Download complete 2024-08-20T21:44:40.0552594Z f4cb80c411d1: Verifying Checksum 2024-08-20T21:44:40.0553057Z f4cb80c411d1: Download complete 2024-08-20T21:44:40.1476742Z c9589ebd9ae7: Verifying Checksum 2024-08-20T21:44:40.1477201Z c9589ebd9ae7: Download complete 2024-08-20T21:44:40.2135944Z dd51c681db14: Download complete 2024-08-20T21:44:40.3185954Z fc983e33a069: Verifying Checksum 2024-08-20T21:44:40.3186389Z fc983e33a069: Download complete 2024-08-20T21:44:40.3848559Z bc705d688c77: Verifying Checksum 2024-08-20T21:44:40.3848999Z bc705d688c77: Download complete 2024-08-20T21:44:40.4527861Z 99f6c1ac7582: Verifying Checksum 2024-08-20T21:44:40.4528287Z 99f6c1ac7582: Download complete 2024-08-20T21:44:40.5240199Z c409d27a5a95: Download complete 2024-08-20T21:44:41.4132837Z 71b614615bcb: Verifying Checksum 2024-08-20T21:44:41.4133306Z 71b614615bcb: Download complete 2024-08-20T21:44:41.4769255Z 314875f5fae4: Download complete 2024-08-20T21:44:41.5443667Z 8a5e90dafefa: Verifying Checksum 2024-08-20T21:44:41.5444181Z 8a5e90dafefa: Download complete 2024-08-20T21:44:41.5925131Z a3a398ca1120: Verifying Checksum 2024-08-20T21:44:41.5925624Z a3a398ca1120: Download complete 2024-08-20T21:44:41.6162683Z e3b946ab3378: Download complete 2024-08-20T21:44:41.6645921Z 14a85a990f71: Verifying Checksum 2024-08-20T21:44:41.6646423Z 14a85a990f71: Download complete 2024-08-20T21:44:41.6848316Z 16d809b167d4: Verifying Checksum 2024-08-20T21:44:41.6848774Z 16d809b167d4: Download complete 2024-08-20T21:44:41.7452924Z b35ca0542723: Verifying Checksum 2024-08-20T21:44:41.7453388Z b35ca0542723: Download complete 2024-08-20T21:44:41.8227120Z bad10ac3a627: Verifying Checksum 2024-08-20T21:44:41.8227590Z bad10ac3a627: Download complete 2024-08-20T21:44:41.8947521Z 943da2164c7a: Verifying Checksum 2024-08-20T21:44:41.9589578Z 2d276441bd8a: Verifying Checksum 2024-08-20T21:44:41.9590039Z 2d276441bd8a: Download complete 2024-08-20T21:44:42.0442008Z a3ef8d7d02be: Download complete 2024-08-20T21:44:42.1131456Z 6aaed7a78749: Verifying Checksum 2024-08-20T21:44:42.1131910Z 6aaed7a78749: Download complete 2024-08-20T21:44:42.1880846Z 944e78083916: Verifying Checksum 2024-08-20T21:44:42.1881360Z 944e78083916: Download complete 2024-08-20T21:44:42.2657780Z 9602de36240c: Verifying Checksum 2024-08-20T21:44:42.2658356Z 9602de36240c: Download complete 2024-08-20T21:44:42.3366851Z 6d3ac49638ec: Verifying Checksum 2024-08-20T21:44:42.3367300Z 6d3ac49638ec: Download complete 2024-08-20T21:44:42.4129568Z c5c5dbfb1dcc: Verifying Checksum 2024-08-20T21:44:42.4130356Z c5c5dbfb1dcc: Download complete 2024-08-20T21:44:42.4887110Z 327bd4a08d3c: Verifying Checksum 2024-08-20T21:44:42.4887866Z 327bd4a08d3c: Download complete 2024-08-20T21:44:42.5638051Z 6c520cbfb159: Verifying Checksum 2024-08-20T21:44:42.5638568Z 6c520cbfb159: Download complete 2024-08-20T21:44:43.0414130Z a8cda4c98926: Verifying Checksum 2024-08-20T21:44:43.0415209Z a8cda4c98926: Download complete 2024-08-20T21:44:43.1519346Z 3d995c530b49: Verifying Checksum 2024-08-20T21:44:43.1519902Z 3d995c530b49: Download complete 2024-08-20T21:44:43.2156047Z 51635df36934: Download complete 2024-08-20T21:44:43.2859260Z 58f9d093cf68: Verifying Checksum 2024-08-20T21:44:43.2859812Z 58f9d093cf68: Download complete 2024-08-20T21:44:43.3423865Z db291b17fdf0: Download complete 2024-08-20T21:44:43.5763152Z 4390932d47dc: Verifying Checksum 2024-08-20T21:44:43.5763795Z 4390932d47dc: Download complete 2024-08-20T21:44:43.6683171Z d959da1ed4a8: Verifying Checksum 2024-08-20T21:44:43.6683639Z d959da1ed4a8: Download complete 2024-08-20T21:44:43.7452382Z 82769435ccd5: Download complete 2024-08-20T21:44:43.8082683Z 50078e7792d3: Verifying Checksum 2024-08-20T21:44:43.8083147Z 50078e7792d3: Download complete 2024-08-20T21:44:43.9005663Z 557fee3a8871: Verifying Checksum 2024-08-20T21:44:43.9006451Z 557fee3a8871: Download complete 2024-08-20T21:44:47.8319969Z 9b1a95eedb83: Verifying Checksum 2024-08-20T21:44:47.8321472Z 9b1a95eedb83: Download complete 2024-08-20T21:44:47.9046874Z f18a0119669a: Verifying Checksum 2024-08-20T21:44:47.9047306Z f18a0119669a: Download complete 2024-08-20T21:44:47.9826815Z 6197a6a036ac: Verifying Checksum 2024-08-20T21:44:47.9827267Z 6197a6a036ac: Download complete 2024-08-20T21:44:48.0495428Z ebdeb237156e: Verifying Checksum 2024-08-20T21:44:48.0496618Z ebdeb237156e: Download complete 2024-08-20T21:44:48.2841818Z 71b614615bcb: Pull complete 2024-08-20T21:44:48.5429334Z f96c8558662d: Pull complete 2024-08-20T21:44:48.7484449Z c81ff9d20d36: Pull complete 2024-08-20T21:44:48.9730580Z 151e9a9d2894: Pull complete 2024-08-20T21:44:49.2125544Z 8538dcd89f0c: Pull complete 2024-08-20T21:44:49.4460468Z cd13007f6f09: Pull complete 2024-08-20T21:44:51.7804592Z 74fd96b6ebd9: Pull complete 2024-08-20T21:44:52.0282107Z ce4e1800db98: Pull complete 2024-08-20T21:44:52.1404423Z 402b067f30bf: Pull complete 2024-08-20T21:44:52.5298857Z 2cebdfe4c5a8: Pull complete 2024-08-20T21:45:05.5493003Z c6e13ff8aa9a: Verifying Checksum 2024-08-20T21:45:05.5493703Z c6e13ff8aa9a: Download complete 2024-08-20T21:45:05.6286469Z 6b068755e588: Verifying Checksum 2024-08-20T21:45:05.6286957Z 6b068755e588: Download complete 2024-08-20T21:45:05.7080749Z 371497d14c4e: Verifying Checksum 2024-08-20T21:45:05.7081199Z 371497d14c4e: Download complete 2024-08-20T21:45:05.7774867Z 4b50cb25a6ac: Verifying Checksum 2024-08-20T21:45:05.7775319Z 4b50cb25a6ac: Download complete 2024-08-20T21:45:05.8360515Z 889d8139c009: Verifying Checksum 2024-08-20T21:45:05.8360943Z 889d8139c009: Download complete 2024-08-20T21:45:05.9098592Z 2bcabd118f67: Verifying Checksum 2024-08-20T21:45:05.9099056Z 2bcabd118f67: Download complete 2024-08-20T21:45:05.9816792Z 1d831c1ea7c7: Verifying Checksum 2024-08-20T21:45:05.9817237Z 1d831c1ea7c7: Download complete 2024-08-20T21:45:06.0543647Z d29d42814800: Verifying Checksum 2024-08-20T21:45:06.0544124Z d29d42814800: Download complete 2024-08-20T21:45:06.1236887Z 10df3ed01753: Download complete 2024-08-20T21:45:06.2025421Z 3b3541d6658c: Download complete 2024-08-20T21:45:06.2773347Z 5e997601a632: Verifying Checksum 2024-08-20T21:45:06.2774093Z 5e997601a632: Download complete 2024-08-20T21:45:06.3469052Z d2924613594f: Verifying Checksum 2024-08-20T21:45:06.3469505Z d2924613594f: Download complete 2024-08-20T21:45:06.4206268Z f66c8294ffd0: Verifying Checksum 2024-08-20T21:45:06.4206720Z f66c8294ffd0: Download complete 2024-08-20T21:45:06.5663284Z 9da0ec93b0d7: Verifying Checksum 2024-08-20T21:45:06.5663807Z 9da0ec93b0d7: Download complete 2024-08-20T21:45:06.6339280Z d476e4b86275: Verifying Checksum 2024-08-20T21:45:06.6339803Z d476e4b86275: Download complete 2024-08-20T21:45:06.7177320Z 397c513e7e07: Verifying Checksum 2024-08-20T21:45:06.7177907Z 397c513e7e07: Download complete 2024-08-20T21:45:06.7865971Z 5d6d58e6432b: Download complete 2024-08-20T21:45:06.8591130Z f9db7741af14: Download complete 2024-08-20T21:45:06.9265599Z c0f5c95392e3: Verifying Checksum 2024-08-20T21:45:06.9266091Z c0f5c95392e3: Download complete 2024-08-20T21:45:07.0286890Z d6d4b368f5e6: Verifying Checksum 2024-08-20T21:45:07.0287667Z d6d4b368f5e6: Download complete 2024-08-20T21:45:07.6152442Z 6e341d8be4cd: Verifying Checksum 2024-08-20T21:45:07.6152934Z 6e341d8be4cd: Download complete 2024-08-20T21:45:25.4661044Z fd5d0d6ba583: Verifying Checksum 2024-08-20T21:45:25.4661566Z fd5d0d6ba583: Download complete 2024-08-20T21:45:32.3707162Z cc0b3fc4f447: Verifying Checksum 2024-08-20T21:45:32.3708871Z cc0b3fc4f447: Download complete 2024-08-20T21:45:43.7776817Z c6e13ff8aa9a: Pull complete 2024-08-20T21:45:44.0031195Z 4f4fb700ef54: Pull complete 2024-08-20T21:45:44.2400484Z e238df05ddcc: Pull complete 2024-08-20T21:45:44.4035615Z a88675bdc723: Pull complete 2024-08-20T21:45:44.5256011Z f4cb80c411d1: Pull complete 2024-08-20T21:45:44.9277715Z c9589ebd9ae7: Pull complete 2024-08-20T21:45:45.1907020Z dd51c681db14: Pull complete 2024-08-20T21:45:45.4342460Z fc983e33a069: Pull complete 2024-08-20T21:45:45.5788963Z bc705d688c77: Pull complete 2024-08-20T21:45:45.8527104Z 99f6c1ac7582: Pull complete 2024-08-20T21:45:45.9881853Z c409d27a5a95: Pull complete 2024-08-20T21:45:47.8595783Z a3a398ca1120: Pull complete 2024-08-20T21:45:48.1368051Z 314875f5fae4: Pull complete 2024-08-20T21:45:48.3664345Z 8a5e90dafefa: Pull complete 2024-08-20T21:45:48.5883768Z e3b946ab3378: Pull complete 2024-08-20T21:45:48.8117056Z 14a85a990f71: Pull complete 2024-08-20T21:45:49.0289353Z 16d809b167d4: Pull complete 2024-08-20T21:45:56.6390465Z 9b1a95eedb83: Pull complete 2024-08-20T21:45:56.7967858Z b35ca0542723: Pull complete 2024-08-20T21:45:56.9167495Z bad10ac3a627: Pull complete 2024-08-20T21:45:57.3142493Z 943da2164c7a: Pull complete 2024-08-20T21:45:57.5944026Z 2d276441bd8a: Pull complete 2024-08-20T21:45:57.8119748Z a3ef8d7d02be: Pull complete 2024-08-20T21:45:58.1058225Z 6aaed7a78749: Pull complete 2024-08-20T21:45:58.3843438Z 944e78083916: Pull complete 2024-08-20T21:45:58.6916572Z 9602de36240c: Pull complete 2024-08-20T21:45:59.2475307Z 6d3ac49638ec: Pull complete 2024-08-20T21:45:59.5186789Z c5c5dbfb1dcc: Pull complete 2024-08-20T21:46:00.1727786Z 327bd4a08d3c: Pull complete 2024-08-20T21:46:00.4238472Z 6c520cbfb159: Pull complete 2024-08-20T21:46:02.0147793Z a8cda4c98926: Pull complete 2024-08-20T21:46:02.0637256Z 3d995c530b49: Pull complete 2024-08-20T21:46:02.1376617Z 51635df36934: Pull complete 2024-08-20T21:46:02.3250557Z 58f9d093cf68: Pull complete 2024-08-20T21:46:02.7506595Z db291b17fdf0: Pull complete 2024-08-20T21:46:03.1446081Z 4390932d47dc: Pull complete 2024-08-20T21:46:03.3766386Z d959da1ed4a8: Pull complete 2024-08-20T21:46:03.5155607Z 82769435ccd5: Pull complete 2024-08-20T21:46:03.6060243Z 50078e7792d3: Pull complete 2024-08-20T21:46:03.6309910Z 557fee3a8871: Pull complete 2024-08-20T21:46:32.1821292Z cc0b3fc4f447: Pull complete 2024-08-20T21:46:32.4767257Z f18a0119669a: Pull complete 2024-08-20T21:46:32.9568383Z 6197a6a036ac: Pull complete 2024-08-20T21:46:33.6151844Z ebdeb237156e: Pull complete 2024-08-20T21:47:20.7917340Z fd5d0d6ba583: Pull complete 2024-08-20T21:47:21.0788939Z 6b068755e588: Pull complete 2024-08-20T21:47:21.3141855Z 371497d14c4e: Pull complete 2024-08-20T21:47:21.9026718Z 4b50cb25a6ac: Pull complete 2024-08-20T21:47:22.5795201Z 889d8139c009: Pull complete 2024-08-20T21:47:22.8284950Z 2bcabd118f67: Pull complete 2024-08-20T21:47:23.4275837Z 1d831c1ea7c7: Pull complete 2024-08-20T21:47:24.0883385Z d29d42814800: Pull complete 2024-08-20T21:47:24.3780933Z 10df3ed01753: Pull complete 2024-08-20T21:47:24.9720661Z 3b3541d6658c: Pull complete 2024-08-20T21:47:25.2083063Z 5e997601a632: Pull complete 2024-08-20T21:47:25.4139909Z d2924613594f: Pull complete 2024-08-20T21:47:25.6105293Z f66c8294ffd0: Pull complete 2024-08-20T21:47:26.0160193Z 9da0ec93b0d7: Pull complete 2024-08-20T21:47:26.2798569Z d476e4b86275: Pull complete 2024-08-20T21:47:26.5023429Z 397c513e7e07: Pull complete 2024-08-20T21:47:26.7163679Z 5d6d58e6432b: Pull complete 2024-08-20T21:47:26.9399997Z f9db7741af14: Pull complete 2024-08-20T21:47:27.1783555Z c0f5c95392e3: Pull complete 2024-08-20T21:47:27.7052555Z d6d4b368f5e6: Pull complete 2024-08-20T21:47:29.4222728Z 6e341d8be4cd: Pull complete 2024-08-20T21:47:30.0693393Z Digest: sha256:ed18f711327455e269271a17448384275684348a472ba289f00904bd14501fa9 2024-08-20T21:47:30.1679176Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:47:30.2139619Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:47:30.2235508Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T21:47:30.2236486Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T21:47:30.2243962Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:30.2244457Z env: 2024-08-20T21:47:30.2244731Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:30.2245081Z ##[endgroup] 2024-08-20T21:47:30.2510164Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T21:47:30.2510963Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T21:47:30.2511671Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T21:47:30.2512340Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:47:30.2518529Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:30.2519003Z env: 2024-08-20T21:47:30.2519271Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:30.2519606Z ##[endgroup] 2024-08-20T21:47:30.5763767Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T21:47:30.5924634Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-08-20T21:47:30.5929678Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-08-20T21:47:30.7081139Z Prepare all required actions 2024-08-20T21:47:30.7081702Z Getting action download info 2024-08-20T21:47:30.8207736Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T21:47:31.1272160Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T21:47:31.2979778Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T21:47:31.2980237Z with: 2024-08-20T21:47:31.2980525Z name: linux-jammy-py3.8-gcc11-build 2024-08-20T21:47:31.2980934Z s3-bucket: gha-artifacts 2024-08-20T21:47:31.2981252Z env: 2024-08-20T21:47:31.2981490Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:31.2981818Z ##[endgroup] 2024-08-20T21:47:31.3165829Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T21:47:31.3166278Z with: 2024-08-20T21:47:31.3166577Z name: linux-jammy-py3.8-gcc11-build 2024-08-20T21:47:31.3166996Z s3-bucket: gha-artifacts 2024-08-20T21:47:31.3167377Z region: us-east-1 2024-08-20T21:47:31.3167655Z env: 2024-08-20T21:47:31.3167912Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:31.3168246Z ##[endgroup] 2024-08-20T21:47:31.7787299Z (node:363933) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T21:47:31.7788033Z 2024-08-20T21:47:31.7788297Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T21:47:31.7789008Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T21:47:31.7789922Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T21:47:31.8439370Z Found 1 objects with prefix pytorch/pytorch/10479310725/linux-jammy-py3.8-gcc11-build/ 2024-08-20T21:47:31.8440438Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T21:47:35.5869023Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T21:47:35.5875398Z Artifact download has finished successfully 2024-08-20T21:47:35.6156386Z ##[group]Run unzip -o artifacts.zip 2024-08-20T21:47:35.6156785Z unzip -o artifacts.zip 2024-08-20T21:47:35.6163093Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:35.6163584Z env: 2024-08-20T21:47:35.6164016Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:35.6164357Z ##[endgroup] 2024-08-20T21:47:35.6202867Z Archive: artifacts.zip 2024-08-20T21:47:35.6203416Z creating: dist/ 2024-08-20T21:47:36.5479295Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl 2024-08-20T21:47:36.5479989Z creating: build/custom_test_artifacts/ 2024-08-20T21:47:36.5480607Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T21:47:36.5481338Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T21:47:36.5482328Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:47:36.5483360Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:47:36.5484331Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T21:47:36.5485300Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:47:36.5486340Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:47:36.5487358Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:47:36.5488528Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:47:36.5489704Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:47:36.5490741Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:47:36.5492014Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:47:36.5493377Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:47:36.5494613Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:47:36.5495805Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:47:36.5497007Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:47:36.5498249Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:47:36.5499477Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:47:36.5500504Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:47:36.5501460Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T21:47:36.5502431Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T21:47:36.5503507Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T21:47:36.5504722Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T21:47:36.5505882Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T21:47:36.5506957Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T21:47:36.5508087Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T21:47:36.5509224Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T21:47:36.5510367Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T21:47:36.5531903Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T21:47:36.5533048Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T21:47:36.5534187Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T21:47:36.5645004Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T21:47:36.5646092Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T21:47:36.5647256Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T21:47:36.5648536Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T21:47:36.5649800Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T21:47:36.5650957Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T21:47:36.5652157Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T21:47:36.5653363Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T21:47:36.5654562Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T21:47:36.5655764Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T21:47:36.5656954Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T21:47:36.5668034Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T21:47:36.5742155Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T21:47:36.5743478Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:47:36.5744634Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:47:36.5745671Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T21:47:36.5746625Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T21:47:36.5747549Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T21:47:36.5748461Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T21:47:36.5749270Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T21:47:36.5750118Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T21:47:36.5856412Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T21:47:36.5914080Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T21:47:36.5914900Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T21:47:36.5915623Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T21:47:36.5916478Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:47:36.5917476Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:47:36.5918440Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T21:47:36.5919410Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:47:36.5920432Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:47:36.5921549Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:47:36.5922707Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:47:36.5923862Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:47:36.5924900Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:47:36.5925936Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:47:36.5927136Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:47:36.5928355Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:47:36.5929546Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:47:36.5930743Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:47:36.5931970Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:47:36.5933177Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:47:36.5934192Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:47:36.5935145Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T21:47:36.5936135Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T21:47:36.5937238Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T21:47:36.5938490Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T21:47:36.5939859Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T21:47:36.5941076Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T21:47:36.5942266Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T21:47:36.5943426Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T21:47:36.5944595Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T21:47:36.5945762Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T21:47:36.5946920Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T21:47:36.5952612Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T21:47:36.6011882Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T21:47:36.6013161Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:47:36.6014302Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:47:36.6015321Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T21:47:36.6016261Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T21:47:36.6017172Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T21:47:36.6018059Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T21:47:36.6018861Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T21:47:36.6019687Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T21:47:36.6064938Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T21:47:36.6065756Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T21:47:36.6066550Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T21:47:36.6067480Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:47:36.6068571Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:47:36.6069598Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T21:47:36.6070658Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:47:36.6071769Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:47:36.6072861Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:47:36.6074081Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:47:36.6075325Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:47:36.6076446Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:47:36.6077551Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:47:36.6078850Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:47:36.6080128Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:47:36.6081403Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:47:36.6082979Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:47:36.6084362Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:47:36.6085666Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:47:36.6086756Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:47:36.6087788Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T21:47:36.6088852Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T21:47:36.6090049Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T21:47:36.6091388Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T21:47:36.6092672Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T21:47:36.6093887Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T21:47:36.6095131Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T21:47:36.6096389Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T21:47:36.6097641Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T21:47:36.6098893Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T21:47:36.6100141Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T21:47:36.6101478Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T21:47:36.6204145Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T21:47:36.6205395Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T21:47:36.6206673Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T21:47:36.6208078Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T21:47:36.6209432Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T21:47:36.6210721Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T21:47:36.6212035Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T21:47:36.6213381Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T21:47:36.6214707Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T21:47:36.6216031Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T21:47:36.6217335Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T21:47:36.6227012Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T21:47:36.6278980Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T21:47:36.6280395Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:47:36.6281897Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:47:36.6283240Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T21:47:36.6284250Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T21:47:36.6285268Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T21:47:36.6286232Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T21:47:36.6287100Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T21:47:36.6287982Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T21:47:36.6381529Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T21:47:36.6422183Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T21:47:36.6422870Z creating: build/lib/ 2024-08-20T21:47:36.6423231Z inflating: build/lib/libclog.a 2024-08-20T21:47:36.6430389Z inflating: build/lib/libpthreadpool.a 2024-08-20T21:47:36.6438322Z inflating: build/lib/libcpuinfo.a 2024-08-20T21:47:36.6445769Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T21:47:36.6509643Z inflating: build/lib/libgtest.a 2024-08-20T21:47:36.6515547Z inflating: build/lib/libittnotify.a 2024-08-20T21:47:36.6592906Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T21:47:36.6661514Z inflating: build/lib/libbenchmark.a 2024-08-20T21:47:36.6687711Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T21:47:36.6748784Z inflating: build/lib/libasmjit.a 2024-08-20T21:47:36.6856301Z inflating: build/lib/libgloo.a 2024-08-20T21:47:36.7273013Z inflating: build/lib/libprotobuf.a 2024-08-20T21:47:36.7291528Z inflating: build/lib/libfmt.a 2024-08-20T21:47:36.7382968Z inflating: build/lib/libc10.so 2024-08-20T21:47:36.7383583Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T21:47:36.7384790Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T21:47:36.7403529Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T21:47:36.7871565Z inflating: build/lib/libprotoc.a 2024-08-20T21:47:36.7887299Z inflating: build/lib/libgmock.a 2024-08-20T21:47:36.7887883Z inflating: build/lib/libgtest_main.a 2024-08-20T21:47:36.7888657Z inflating: build/lib/libbenchmark_main.a 2024-08-20T21:47:36.7906296Z inflating: build/lib/libnnpack.a 2024-08-20T21:47:37.6844871Z inflating: build/lib/libdnnl.a 2024-08-20T21:47:37.7360419Z inflating: build/lib/libtensorpipe.a 2024-08-20T21:47:37.8329149Z inflating: build/lib/libfbgemm.a 2024-08-20T21:47:37.8594505Z inflating: build/lib/libkineto.a 2024-08-20T21:47:37.8595078Z inflating: build/lib/libgmock_main.a 2024-08-20T21:47:37.8633486Z inflating: build/lib/libonnx_proto.a 2024-08-20T21:47:37.8803656Z inflating: build/lib/libXNNPACK.a 2024-08-20T21:47:37.9448594Z inflating: build/lib/libonnx.a 2024-08-20T21:47:40.1331331Z inflating: build/lib/libtorch_cpu.so 2024-08-20T21:47:40.1332255Z inflating: build/lib/libtorch.so 2024-08-20T21:47:40.1336339Z inflating: build/lib/libunbox_lib.a 2024-08-20T21:47:40.1339864Z inflating: build/lib/libshm.so 2024-08-20T21:47:40.1359155Z inflating: build/lib/libjitbackend_test.so 2024-08-20T21:47:40.1426847Z inflating: build/lib/libtorchbind_test.so 2024-08-20T21:47:40.1449889Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T21:47:40.1473414Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T21:47:40.3244308Z inflating: build/lib/libtorch_python.so 2024-08-20T21:47:40.3278746Z inflating: build/lib/libnnapi_backend.so 2024-08-20T21:47:40.3279234Z creating: build/bin/ 2024-08-20T21:47:40.3279662Z creating: build/bin/CMakeFiles/ 2024-08-20T21:47:40.3280248Z inflating: build/bin/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:47:40.3281143Z extracting: build/bin/CMakeFiles/progress.marks 2024-08-20T21:47:40.3281903Z inflating: build/bin/Makefile 2024-08-20T21:47:40.3282346Z inflating: build/bin/cmake_install.cmake 2024-08-20T21:47:40.3282833Z inflating: build/bin/CTestTestfile.cmake 2024-08-20T21:47:40.3329779Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T21:47:40.3382170Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T21:47:40.3432597Z inflating: build/bin/c10_Device_test 2024-08-20T21:47:40.3486533Z inflating: build/bin/c10_Scalar_test 2024-08-20T21:47:40.3545284Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T21:47:40.3593172Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T21:47:40.3643107Z inflating: build/bin/c10_SymInt_test 2024-08-20T21:47:40.3697833Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T21:47:40.3753237Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T21:47:40.3808482Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T21:47:40.3858747Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T21:47:40.3912055Z inflating: build/bin/c10_Bitset_test 2024-08-20T21:47:40.3981896Z inflating: build/bin/c10_cow_test 2024-08-20T21:47:40.4032406Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T21:47:40.4083472Z inflating: build/bin/c10_Half_test 2024-08-20T21:47:40.4139933Z inflating: build/bin/c10_LeftRight_test 2024-08-20T21:47:40.4196206Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T21:47:40.4245290Z inflating: build/bin/c10_Synchronized_test 2024-08-20T21:47:40.4302106Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T21:47:40.4354733Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T21:47:40.4406618Z inflating: build/bin/c10_TypeList_test 2024-08-20T21:47:40.4459110Z inflating: build/bin/c10_accumulate_test 2024-08-20T21:47:40.4508445Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T21:47:40.4563408Z inflating: build/bin/c10_bfloat16_test 2024-08-20T21:47:40.4612623Z inflating: build/bin/c10_bit_cast_test 2024-08-20T21:47:40.4669369Z inflating: build/bin/c10_complex_math_test 2024-08-20T21:47:40.4723679Z inflating: build/bin/c10_complex_test 2024-08-20T21:47:40.4777274Z inflating: build/bin/c10_exception_test 2024-08-20T21:47:40.4827212Z inflating: build/bin/c10_flags_test 2024-08-20T21:47:40.4877586Z inflating: build/bin/c10_generic_math_test 2024-08-20T21:47:40.4930746Z inflating: build/bin/c10_lazy_test 2024-08-20T21:47:40.4982061Z inflating: build/bin/c10_irange_test 2024-08-20T21:47:40.5136673Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T21:47:40.5195032Z inflating: build/bin/c10_logging_test 2024-08-20T21:47:40.5269264Z inflating: build/bin/c10_optional_test 2024-08-20T21:47:40.5322968Z inflating: build/bin/c10_registry_test 2024-08-20T21:47:40.5384051Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T21:47:40.5526494Z inflating: build/bin/c10_small_vector_test 2024-08-20T21:47:40.5579035Z inflating: build/bin/c10_ssize_test 2024-08-20T21:47:40.5630185Z inflating: build/bin/c10_string_util_test 2024-08-20T21:47:40.5688718Z inflating: build/bin/c10_string_view_test 2024-08-20T21:47:40.5740018Z inflating: build/bin/c10_tempfile_test 2024-08-20T21:47:40.5787107Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T21:47:40.5841639Z inflating: build/bin/c10_typeid_test 2024-08-20T21:47:40.6273694Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T21:47:40.6703510Z inflating: build/bin/protoc 2024-08-20T21:47:40.7034755Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T21:47:40.7354447Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T21:47:40.7701758Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T21:47:40.7754923Z inflating: build/bin/BackoffTest 2024-08-20T21:47:40.7807852Z inflating: build/bin/FileStoreTest 2024-08-20T21:47:40.7864831Z inflating: build/bin/TCPStoreTest 2024-08-20T21:47:40.7918593Z inflating: build/bin/HashStoreTest 2024-08-20T21:47:40.7920914Z inflating: build/bin/example_allreduce 2024-08-20T21:47:40.7987850Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T21:47:40.8041871Z inflating: build/bin/test_dist_autograd 2024-08-20T21:47:40.8043912Z inflating: build/bin/parallel_benchmark 2024-08-20T21:47:40.8111586Z inflating: build/bin/test_cpp_rpc 2024-08-20T21:47:40.8120023Z inflating: build/bin/aot_model_compiler_test 2024-08-20T21:47:40.8186251Z inflating: build/bin/test_mobile_nnc 2024-08-20T21:47:40.8508786Z inflating: build/bin/test_lazy 2024-08-20T21:47:40.8564319Z inflating: build/bin/static_runtime_bench 2024-08-20T21:47:40.8637638Z inflating: build/bin/Dict_test 2024-08-20T21:47:40.8689901Z inflating: build/bin/Dimname_test 2024-08-20T21:47:40.8931851Z inflating: build/bin/static_runtime_test 2024-08-20T21:47:40.8988897Z inflating: build/bin/NamedTensor_test 2024-08-20T21:47:40.9051512Z inflating: build/bin/MaybeOwned_test 2024-08-20T21:47:40.9111546Z inflating: build/bin/apply_utils_test 2024-08-20T21:47:40.9168964Z inflating: build/bin/atest 2024-08-20T21:47:40.9231791Z inflating: build/bin/basic 2024-08-20T21:47:40.9286199Z inflating: build/bin/broadcast_test 2024-08-20T21:47:40.9336856Z inflating: build/bin/cpu_allocator_test 2024-08-20T21:47:40.9395336Z inflating: build/bin/cpu_generator_test 2024-08-20T21:47:40.9446696Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T21:47:40.9538022Z inflating: build/bin/cpu_rng_test 2024-08-20T21:47:40.9588532Z inflating: build/bin/dispatch_key_set_test 2024-08-20T21:47:40.9638486Z inflating: build/bin/dlconvertor_test 2024-08-20T21:47:40.9696945Z inflating: build/bin/extension_backend_test 2024-08-20T21:47:40.9751200Z inflating: build/bin/half_test 2024-08-20T21:47:41.0826755Z inflating: build/bin/test_api 2024-08-20T21:47:41.0876540Z inflating: build/bin/lazy_tensor_test 2024-08-20T21:47:41.0929722Z inflating: build/bin/math_kernel_test 2024-08-20T21:47:41.1022378Z inflating: build/bin/ivalue_test 2024-08-20T21:47:41.1075694Z inflating: build/bin/memory_format_test 2024-08-20T21:47:41.1127072Z inflating: build/bin/operator_name_test 2024-08-20T21:47:41.1181087Z inflating: build/bin/memory_overlapping_test 2024-08-20T21:47:41.1236764Z inflating: build/bin/native_test 2024-08-20T21:47:41.1290109Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T21:47:41.1341175Z inflating: build/bin/operators_test 2024-08-20T21:47:41.1392767Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T21:47:41.1459518Z inflating: build/bin/pow_test 2024-08-20T21:47:41.1508950Z inflating: build/bin/reduce_ops_test 2024-08-20T21:47:41.1564943Z inflating: build/bin/quantized_test 2024-08-20T21:47:41.1617134Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T21:47:41.1672328Z inflating: build/bin/scalar_tensor_test 2024-08-20T21:47:41.1723875Z inflating: build/bin/StorageUtils_test 2024-08-20T21:47:41.1782116Z inflating: build/bin/scalar_test 2024-08-20T21:47:41.1834449Z inflating: build/bin/stride_properties_test 2024-08-20T21:47:41.1912103Z inflating: build/bin/tensor_iterator_test 2024-08-20T21:47:41.1966203Z inflating: build/bin/type_ptr_test 2024-08-20T21:47:41.2019321Z inflating: build/bin/test_parallel 2024-08-20T21:47:41.2021340Z inflating: build/bin/thread_init_test 2024-08-20T21:47:41.2082156Z inflating: build/bin/type_test 2024-08-20T21:47:41.2135560Z inflating: build/bin/undefined_tensor_test 2024-08-20T21:47:41.2136588Z inflating: build/bin/verify_api_visibility 2024-08-20T21:47:41.2205606Z inflating: build/bin/legacy_vmap_test 2024-08-20T21:47:41.2257507Z inflating: build/bin/weakref_test 2024-08-20T21:47:41.2309074Z inflating: build/bin/wrapdim_test 2024-08-20T21:47:41.2367254Z inflating: build/bin/IListRef_test 2024-08-20T21:47:41.2419761Z inflating: build/bin/xla_tensor_test 2024-08-20T21:47:41.2523657Z inflating: build/bin/List_test 2024-08-20T21:47:41.2641063Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T21:47:41.2735615Z inflating: build/bin/kernel_function_test 2024-08-20T21:47:41.2858306Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T21:47:41.2922905Z inflating: build/bin/KernelFunction_test 2024-08-20T21:47:41.3022614Z inflating: build/bin/kernel_lambda_test 2024-08-20T21:47:41.3073760Z inflating: build/bin/CppSignature_test 2024-08-20T21:47:41.3132076Z inflating: build/bin/kernel_stackbased_test 2024-08-20T21:47:41.3225167Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T21:47:41.3273823Z inflating: build/bin/op_allowlist_test 2024-08-20T21:47:41.3338014Z inflating: build/bin/inline_container_test 2024-08-20T21:47:41.3633840Z inflating: build/bin/op_registration_test 2024-08-20T21:47:41.3688667Z inflating: build/bin/backend_fallback_test 2024-08-20T21:47:41.3741250Z inflating: build/bin/test_edge_op_registration 2024-08-20T21:47:41.3754287Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T21:47:41.3758228Z inflating: build/bin/torch_shm_manager 2024-08-20T21:47:41.4512962Z inflating: build/bin/test_tensorexpr 2024-08-20T21:47:41.5067130Z inflating: build/bin/test_jit 2024-08-20T21:47:41.5067893Z creating: .additional_ci_files/ 2024-08-20T21:47:41.5122765Z inflating: .additional_ci_files/test-times.json 2024-08-20T21:47:41.5348069Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T21:47:41.5425989Z ##[group]Run rm artifacts.zip 2024-08-20T21:47:41.5426343Z rm artifacts.zip 2024-08-20T21:47:41.5432149Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:41.5432607Z env: 2024-08-20T21:47:41.5432839Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:41.5433156Z ##[endgroup] 2024-08-20T21:47:41.5798749Z ##[group]Run df -H 2024-08-20T21:47:41.5799033Z df -H 2024-08-20T21:47:41.5804529Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:41.5805018Z env: 2024-08-20T21:47:41.5805262Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:41.5805588Z ##[endgroup] 2024-08-20T21:47:41.5842786Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T21:47:41.5843317Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T21:47:41.5843790Z tmpfs 50G 0 50G 0% /dev/shm 2024-08-20T21:47:41.5844248Z tmpfs 20G 652k 20G 1% /run 2024-08-20T21:47:41.5844722Z /dev/nvme0n1p1 215G 39G 177G 18% / 2024-08-20T21:47:41.5845150Z tmpfs 50G 29k 50G 1% /tmp 2024-08-20T21:47:41.5845620Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T21:47:41.5885827Z Prepare all required actions 2024-08-20T21:47:41.5886258Z Getting action download info 2024-08-20T21:47:41.7140999Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T21:47:41.7141622Z with: 2024-08-20T21:47:41.7141842Z env: 2024-08-20T21:47:41.7142097Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:41.7142424Z ##[endgroup] 2024-08-20T21:47:41.7392038Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T21:47:41.7392455Z with: 2024-08-20T21:47:41.7392697Z name: td_results 2024-08-20T21:47:41.7392974Z s3-bucket: gha-artifacts 2024-08-20T21:47:41.7393293Z region: us-east-1 2024-08-20T21:47:41.7393559Z env: 2024-08-20T21:47:41.7393804Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:41.7394103Z ##[endgroup] 2024-08-20T21:47:42.2013341Z (node:363954) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T21:47:42.2014137Z 2024-08-20T21:47:42.2014459Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T21:47:42.2015196Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T21:47:42.2016092Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T21:47:42.2623192Z Found 0 objects with prefix pytorch/pytorch/10479310725/td_results/ 2024-08-20T21:47:42.2628283Z Artifact download has finished successfully 2024-08-20T21:47:42.2804403Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T21:47:42.2804876Z mkdir -p .additional_ci_files 2024-08-20T21:47:42.2805434Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T21:47:42.2811168Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:42.2811620Z env: 2024-08-20T21:47:42.2811848Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:42.2812161Z ##[endgroup] 2024-08-20T21:47:42.3160031Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T21:47:42.3172385Z ##[error]Process completed with exit code 1. 2024-08-20T21:47:42.3261059Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T21:47:42.3261498Z .github/scripts/parse_ref.py 2024-08-20T21:47:42.3267302Z shell: /usr/bin/bash -e {0} 2024-08-20T21:47:42.3267658Z env: 2024-08-20T21:47:42.3267917Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:42.3268252Z ##[endgroup] 2024-08-20T21:47:42.3557981Z Prepare all required actions 2024-08-20T21:47:42.3768993Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T21:47:42.3769443Z with: 2024-08-20T21:47:42.3769998Z github-token: *** 2024-08-20T21:47:42.3770290Z env: 2024-08-20T21:47:42.3770535Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:42.3770865Z ##[endgroup] 2024-08-20T21:47:42.3868432Z ##[group]Run set -eux 2024-08-20T21:47:42.3868751Z set -eux 2024-08-20T21:47:42.3869338Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T21:47:42.3874830Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:42.3875306Z env: 2024-08-20T21:47:42.3875565Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:42.3876077Z GITHUB_TOKEN: *** 2024-08-20T21:47:42.3876369Z ##[endgroup] 2024-08-20T21:47:42.3898231Z + python3 .github/scripts/get_workflow_job_id.py 10479310725 i-0a9f60968c6bae802 2024-08-20T21:47:43.4958671Z setting job-id=29025551430 2024-08-20T21:47:43.4960128Z setting job-name=linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:43.5201294Z Prepare all required actions 2024-08-20T21:47:43.5201751Z Getting action download info 2024-08-20T21:47:43.6592628Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T21:47:43.6593042Z with: 2024-08-20T21:47:43.6593474Z github-token: *** 2024-08-20T21:47:43.6614308Z test-matrix: {"include": [{"config": "inductor_avx512", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_avx512", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_amp_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_torchbench_cpu_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.24xl.spr-metal"}, {"config": "inductor_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "inductor_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_huggingface_freezing_avx2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}]} 2024-08-20T21:47:43.6635338Z job-name: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:43.6636186Z env: 2024-08-20T21:47:43.6636431Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:43.6636747Z ##[endgroup] 2024-08-20T21:47:43.6807684Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T21:47:43.6808241Z with: 2024-08-20T21:47:43.6808503Z shell: bash 2024-08-20T21:47:43.6808792Z timeout_minutes: 10 2024-08-20T21:47:43.6809117Z max_attempts: 5 2024-08-20T21:47:43.6809408Z retry_wait_seconds: 30 2024-08-20T21:47:43.6810561Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T21:47:43.6811935Z polling_interval_seconds: 1 2024-08-20T21:47:43.6812307Z warning_on_retry: true 2024-08-20T21:47:43.6812630Z continue_on_error: false 2024-08-20T21:47:43.6812963Z env: 2024-08-20T21:47:43.6813226Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:43.6813871Z GITHUB_TOKEN: *** 2024-08-20T21:47:43.6814140Z ##[endgroup] 2024-08-20T21:47:43.7519103Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T21:47:43.9862326Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T21:47:44.0019929Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2024-08-20T21:47:44.0021622Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2024-08-20T21:47:44.0141755Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2024-08-20T21:47:44.0148109Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-20T21:47:44.0152289Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-20T21:47:44.0157895Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-20T21:47:44.7356867Z Command completed after 1 attempt(s). 2024-08-20T21:47:44.7434056Z ##[group]Run set -x 2024-08-20T21:47:44.7434397Z set -x 2024-08-20T21:47:44.7434699Z  2024-08-20T21:47:44.7435255Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T21:47:44.7436041Z # in runner workspace 2024-08-20T21:47:44.7436559Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T21:47:44.7443135Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:44.7443658Z env: 2024-08-20T21:47:44.7443920Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:44.7444268Z ##[endgroup] 2024-08-20T21:47:44.7471873Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T21:47:44.7798902Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T21:47:44.7799482Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T21:47:44.7799935Z echo "Job name: ${JOB_NAME}" 2024-08-20T21:47:44.7800321Z  2024-08-20T21:47:44.7800852Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T21:47:44.7801815Z # in runner workspace 2024-08-20T21:47:44.7802400Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T21:47:44.7803056Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T21:47:44.7803513Z  --job-name "${JOB_NAME}" \ 2024-08-20T21:47:44.7824077Z  --test-matrix "{"include": [{"config": "inductor_avx512", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_avx512", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_amp_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_torchbench_cpu_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.24xl.spr-metal"}, {"config": "inductor_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "inductor_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_huggingface_freezing_avx2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}]}" \ 2024-08-20T21:47:44.7844979Z  --selected-test-configs "" \ 2024-08-20T21:47:44.7845432Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T21:47:44.7845828Z  --tag "${TAG}" \ 2024-08-20T21:47:44.7846213Z  --event-name "${EVENT_NAME}" \ 2024-08-20T21:47:44.7846650Z  --schedule "${SCHEDULE}" \ 2024-08-20T21:47:44.7847071Z  --branch "${HEAD_BRANCH}" 2024-08-20T21:47:44.7852664Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:44.7853245Z env: 2024-08-20T21:47:44.7853511Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:44.7854053Z GITHUB_TOKEN: *** 2024-08-20T21:47:44.7854877Z JOB_NAME: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:44.7855787Z PR_NUMBER: 2024-08-20T21:47:44.7856198Z TAG: ciflow/inductor/133712 2024-08-20T21:47:44.7856545Z EVENT_NAME: push 2024-08-20T21:47:44.7856820Z SCHEDULE: 2024-08-20T21:47:44.7857084Z HEAD_BRANCH: 2024-08-20T21:47:44.7857356Z ##[endgroup] 2024-08-20T21:47:44.7879892Z Workflow: inductor 2024-08-20T21:47:44.7881076Z Job name: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:45.0581797Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T21:47:45.2287459Z ##[group]Run echo "Filtered matrix:" 2024-08-20T21:47:45.2287954Z echo "Filtered matrix:" 2024-08-20T21:47:45.2307588Z echo "{"include": [{"config": "inductor_avx512", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_avx512", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_amp_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_torchbench_cpu_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.24xl.spr-metal"}, {"config": "inductor_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "inductor_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_huggingface_freezing_avx2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}]}" 2024-08-20T21:47:45.2328384Z  2024-08-20T21:47:45.2328628Z echo 2024-08-20T21:47:45.2328992Z echo "Is the current job unstable? False" 2024-08-20T21:47:45.2329439Z  2024-08-20T21:47:45.2329686Z echo 2024-08-20T21:47:45.2330182Z echo "Is keep-going label set? False" 2024-08-20T21:47:45.2330609Z  2024-08-20T21:47:45.2330860Z echo 2024-08-20T21:47:45.2331151Z echo "Renabled issues? " 2024-08-20T21:47:45.2336782Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:45.2337267Z env: 2024-08-20T21:47:45.2337523Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:45.2337935Z ##[endgroup] 2024-08-20T21:47:45.2443741Z Filtered matrix: 2024-08-20T21:47:45.2470840Z {include: [{config: inductor_avx512, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: inductor_avx512, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_huggingface_freezing, shard: 1, num_shards: 1, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_timm_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_timm_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_torchbench_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_torchbench_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_huggingface_amp_freezing, shard: 1, num_shards: 1, runner: amz2023.linux.16xlarge.spr}, {config: cpu_inductor_timm_amp_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.16xlarge.spr}, {config: cpu_inductor_timm_amp_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.16xlarge.spr}, {config: cpu_inductor_torchbench_amp_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.16xlarge.spr}, {config: cpu_inductor_torchbench_amp_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.16xlarge.spr}, {config: dynamic_cpu_inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_huggingface_freezing, shard: 1, num_shards: 1, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_timm_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_timm_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_torchbench_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_torchbench_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_torchbench_amp_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_torchbench_amp_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_aot_inductor_torchbench_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_aot_inductor_torchbench_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_aot_inductor_torchbench_amp_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_aot_inductor_torchbench_amp_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: inductor_torchbench_cpu_smoketest_perf, shard: 1, num_shards: 1, runner: amz2023.linux.24xl.spr-metal}, {config: inductor_avx2, shard: 1, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: inductor_avx2, shard: 2, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_huggingface_freezing_avx2, shard: 1, num_shards: 1, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_torchbench_freezing_avx2, shard: 1, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_torchbench_freezing_avx2, shard: 2, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_timm_freezing_avx2, shard: 1, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_timm_freezing_avx2, shard: 2, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}]} 2024-08-20T21:47:45.2492868Z 2024-08-20T21:47:45.2493020Z Is the current job unstable? False 2024-08-20T21:47:45.2493293Z 2024-08-20T21:47:45.2493521Z Is keep-going label set? False 2024-08-20T21:47:45.2493768Z 2024-08-20T21:47:45.2493888Z Renabled issues? 2024-08-20T21:47:45.2589685Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T21:47:45.2590482Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T21:47:45.2596032Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:45.2596533Z env: 2024-08-20T21:47:45.2596778Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:45.2597109Z JOB_TIMEOUT: 240 2024-08-20T21:47:45.2597393Z ##[endgroup] 2024-08-20T21:47:45.2779614Z ##[group]Run set -x 2024-08-20T21:47:45.2780002Z set -x 2024-08-20T21:47:45.2780284Z  2024-08-20T21:47:45.2780649Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T21:47:45.2781174Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T21:47:45.2781731Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T21:47:45.2782229Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T21:47:45.2782627Z else 2024-08-20T21:47:45.2782936Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T21:47:45.2783343Z fi 2024-08-20T21:47:45.2783595Z  2024-08-20T21:47:45.2784050Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T21:47:45.2784803Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T21:47:45.2785586Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T21:47:45.2786180Z # shellcheck disable=SC2086,SC2090 2024-08-20T21:47:45.2786620Z container_name=$(docker run \ 2024-08-20T21:47:45.2787024Z  ${GPU_FLAG:-} \ 2024-08-20T21:47:45.2787378Z  -e BUILD_ENVIRONMENT \ 2024-08-20T21:47:45.2787763Z  -e PR_NUMBER \ 2024-08-20T21:47:45.2788093Z  -e GITHUB_ACTIONS \ 2024-08-20T21:47:45.2788470Z  -e GITHUB_REPOSITORY \ 2024-08-20T21:47:45.2788969Z  -e GITHUB_WORKFLOW \ 2024-08-20T21:47:45.2789331Z  -e GITHUB_JOB \ 2024-08-20T21:47:45.2789651Z  -e GITHUB_RUN_ID \ 2024-08-20T21:47:45.2790005Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T21:47:45.2790386Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T21:47:45.2790759Z  -e JOB_ID \ 2024-08-20T21:47:45.2791059Z  -e JOB_NAME \ 2024-08-20T21:47:45.2791387Z  -e BASE_SHA \ 2024-08-20T21:47:45.2791864Z  -e BRANCH \ 2024-08-20T21:47:45.2792188Z  -e SHA1 \ 2024-08-20T21:47:45.2792516Z  -e AWS_DEFAULT_REGION \ 2024-08-20T21:47:45.2792902Z  -e IN_WHEEL_TEST \ 2024-08-20T21:47:45.2793258Z  -e SHARD_NUMBER \ 2024-08-20T21:47:45.2793596Z  -e TEST_CONFIG \ 2024-08-20T21:47:45.2793941Z  -e NUM_TEST_SHARDS \ 2024-08-20T21:47:45.2794314Z  -e REENABLED_ISSUES \ 2024-08-20T21:47:45.2794710Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T21:47:45.2795110Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T21:47:45.2795500Z  -e TEST_SHOWLOCALS \ 2024-08-20T21:47:45.2795873Z  -e NO_TEST_TIMEOUT \ 2024-08-20T21:47:45.2796230Z  -e NO_TD \ 2024-08-20T21:47:45.2796540Z  -e TD_DISTRIBUTED \ 2024-08-20T21:47:45.2796904Z  -e PR_LABELS \ 2024-08-20T21:47:45.2797292Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T21:47:45.2797730Z  -e SCCACHE_BUCKET \ 2024-08-20T21:47:45.2798112Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T21:47:45.2798508Z  -e XLA_CUDA \ 2024-08-20T21:47:45.2798893Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T21:47:45.2799376Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T21:47:45.2799888Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T21:47:45.2800393Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T21:47:45.2800847Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T21:47:45.2801368Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T21:47:45.2801863Z  -e DASHBOARD_TAG \ 2024-08-20T21:47:45.2802336Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T21:47:45.2802890Z  --security-opt seccomp=unconfined \ 2024-08-20T21:47:45.2803341Z  --cap-add=SYS_PTRACE \ 2024-08-20T21:47:45.2803749Z  --ipc=host \ 2024-08-20T21:47:45.2804101Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T21:47:45.2804513Z  --tty \ 2024-08-20T21:47:45.2804808Z  --detach \ 2024-08-20T21:47:45.2805273Z  --name="${container_name}" \ 2024-08-20T21:47:45.2805690Z  --user jenkins \ 2024-08-20T21:47:45.2806169Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T21:47:45.2806727Z  -w /var/lib/jenkins/workspace \ 2024-08-20T21:47:45.2807140Z  "${DOCKER_IMAGE}" 2024-08-20T21:47:45.2807476Z ) 2024-08-20T21:47:45.2807866Z # Propagate download.pytorch.org IP to container 2024-08-20T21:47:45.2808804Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T21:47:45.2809798Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T21:47:45.2810714Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T21:47:45.2816596Z shell: /usr/bin/bash -e {0} 2024-08-20T21:47:45.2816934Z env: 2024-08-20T21:47:45.2817192Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:45.2817604Z BUILD_ENVIRONMENT: linux-jammy-py3.8-gcc11-build 2024-08-20T21:47:45.2818072Z PR_NUMBER: 2024-08-20T21:47:45.2818382Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T21:47:45.2818772Z GITHUB_WORKFLOW: inductor 2024-08-20T21:47:45.2819110Z GITHUB_JOB: test 2024-08-20T21:47:45.2819406Z GITHUB_RUN_ID: 10479310725 2024-08-20T21:47:45.2819748Z GITHUB_RUN_NUMBER: 88082 2024-08-20T21:47:45.2820072Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T21:47:45.2820384Z JOB_ID: 29025551430 2024-08-20T21:47:45.2821208Z JOB_NAME: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:45.2822103Z BRANCH: 2024-08-20T21:47:45.2822412Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:47:45.2822925Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:47:45.2823492Z TEST_CONFIG: dynamic_cpu_aot_inductor_torchbench_freezing 2024-08-20T21:47:45.2823981Z SHARD_NUMBER: 1 2024-08-20T21:47:45.2824263Z NUM_TEST_SHARDS: 2 2024-08-20T21:47:45.2824567Z REENABLED_ISSUES: 2024-08-20T21:47:45.2824881Z CONTINUE_THROUGH_ERROR: False 2024-08-20T21:47:45.2825255Z VERBOSE_TEST_LOGS: False 2024-08-20T21:47:45.2825583Z TEST_SHOWLOCALS: False 2024-08-20T21:47:45.2825909Z NO_TEST_TIMEOUT: False 2024-08-20T21:47:45.2826223Z NO_TD: False 2024-08-20T21:47:45.2826494Z TD_DISTRIBUTED: False 2024-08-20T21:47:45.2827003Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T21:47:45.2827582Z SCCACHE_S3_KEY_PREFIX: inductor 2024-08-20T21:47:45.2827920Z SHM_SIZE: 1g 2024-08-20T21:47:45.2828874Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:47:45.2829909Z XLA_CUDA: 2024-08-20T21:47:45.2830346Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:47:45.2830920Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T21:47:45.2831310Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T21:47:45.2831665Z DASHBOARD_TAG: 2024-08-20T21:47:45.2832133Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T21:47:45.2832472Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T21:47:45.2832804Z ##[endgroup] 2024-08-20T21:47:45.2853993Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == \m\u\l\t\i\g\p\u ]] 2024-08-20T21:47:45.2854818Z + [[ linux-jammy-py3.8-gcc11-build == *onnx* ]] 2024-08-20T21:47:45.2855274Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T21:47:45.2863045Z +++ nproc --ignore=2 2024-08-20T21:47:45.2878369Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=46 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310725 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:49:12.8322207Z + container_name=b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T21:49:12.8325816Z + grep download.pytorch.org /etc/hosts 2024-08-20T21:49:12.8327695Z + docker exec -i b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T21:49:12.9724959Z + echo DOCKER_CONTAINER_ID=b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T21:49:12.9728424Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl 2024-08-20T21:49:12.9730891Z + docker exec -t b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T21:49:13.3442728Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T21:49:13.7665019Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T21:49:13.7667080Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T21:49:13.7669100Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T21:49:13.7670976Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T21:49:13.7673342Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T21:49:13.7678657Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.12.1) 2024-08-20T21:49:13.7692660Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T21:49:13.7711420Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T21:49:13.7725437Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.21.2) 2024-08-20T21:49:13.8086033Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T21:49:14.4504850Z Installing collected packages: torch 2024-08-20T21:49:21.8194426Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T21:49:21.8195988Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T21:49:21.8196753Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T21:49:21.8813779Z + export TERM=vt100 2024-08-20T21:49:21.8814485Z + TERM=vt100 2024-08-20T21:49:21.8814818Z ++ dirname .ci/pytorch/test.sh 2024-08-20T21:49:21.8821720Z + source .ci/pytorch/common.sh 2024-08-20T21:49:21.8825368Z +++ dirname .ci/pytorch/common.sh 2024-08-20T21:49:21.8835927Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T21:49:21.8836555Z +++ declare -f -t trap_add 2024-08-20T21:49:21.8837277Z ++ set -ex 2024-08-20T21:49:21.8837682Z ++ [[ linux-jammy-py3.8-gcc11-build == *rocm* ]] 2024-08-20T21:49:21.8840049Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T21:49:21.8840677Z + [[ linux-jammy-py3.8-gcc11-build != *rocm* ]] 2024-08-20T21:49:21.8841226Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T21:49:21.8854172Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T21:49:21.8855209Z + trap_add cleanup_workspace EXIT 2024-08-20T21:49:21.8855672Z + trap_add_cmd=cleanup_workspace 2024-08-20T21:49:21.8856245Z + shift 2024-08-20T21:49:21.8856517Z + for trap_add_name in "$@" 2024-08-20T21:49:21.8859905Z +++ trap -p EXIT 2024-08-20T21:49:21.8860528Z ++ eval 'extract_trap_cmd ' 2024-08-20T21:49:21.8860915Z +++ extract_trap_cmd 2024-08-20T21:49:21.8861276Z +++ printf '%s\n' '' 2024-08-20T21:49:21.8861663Z ++ printf '%s\n' cleanup_workspace 2024-08-20T21:49:21.8862290Z + trap -- ' 2024-08-20T21:49:21.8862608Z cleanup_workspace' EXIT 2024-08-20T21:49:21.8863076Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T21:49:22.3278142Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T21:49:22.3289881Z + echo 'Environment variables:' 2024-08-20T21:49:22.3290266Z Environment variables: 2024-08-20T21:49:22.3290585Z + env 2024-08-20T21:49:22.3296903Z INSTALLED_DB=yes 2024-08-20T21:49:22.3297470Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:49:22.3298031Z CONTINUE_THROUGH_ERROR=False 2024-08-20T21:49:22.3298524Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11-build 2024-08-20T21:49:22.3298974Z HOSTNAME=b5c5ea11556a 2024-08-20T21:49:22.3302212Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3303092Z GITHUB_ACTION=__self 2024-08-20T21:49:22.3303438Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T21:49:22.3303815Z GITHUB_RUN_NUMBER=88082 2024-08-20T21:49:22.3304263Z TEST_CONFIG=dynamic_cpu_aot_inductor_torchbench_freezing 2024-08-20T21:49:22.3304777Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T21:49:22.3305259Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T21:49:22.3305664Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T21:49:22.3306114Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T21:49:22.3306525Z GITHUB_REF_TYPE=tag 2024-08-20T21:49:22.3306828Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T21:49:22.3307246Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:22.3307683Z XLA_CUDA= 2024-08-20T21:49:22.3308169Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T21:49:22.3309896Z *** 2024-08-20T21:49:22.3310175Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T21:49:22.3310518Z GITHUB_ACTIONS=true 2024-08-20T21:49:22.3310891Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:22.3311401Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:22.3312214Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T21:49:22.3312933Z UCC_HOME=/usr 2024-08-20T21:49:22.3313222Z VERBOSE_TEST_LOGS=False 2024-08-20T21:49:22.3313586Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T21:49:22.3313989Z SHARD_NUMBER=1 2024-08-20T21:49:22.3314284Z GITHUB_REF_PROTECTED=false 2024-08-20T21:49:22.3314623Z HOME=/var/lib/jenkins 2024-08-20T21:49:22.3314976Z GITHUB_API_URL=https://api.github.com 2024-08-20T21:49:22.3315402Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T21:49:22.3315776Z UCX_COMMIT= 2024-08-20T21:49:22.3316056Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T21:49:22.3316423Z NUM_TEST_SHARDS=2 2024-08-20T21:49:22.3316694Z UCX_HOME=/usr 2024-08-20T21:49:22.3317754Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3319245Z JOB_NAME=linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:49:22.3320681Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3321940Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T21:49:22.3322614Z GITHUB_EVENT_NAME=push 2024-08-20T21:49:22.3322932Z DASHBOARD_TAG= 2024-08-20T21:49:22.3323223Z GITHUB_RUN_ID=10479310725 2024-08-20T21:49:22.3324160Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3325095Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T21:49:22.3325580Z PR_NUMBER= 2024-08-20T21:49:22.3325843Z DESIRED_CUDA= 2024-08-20T21:49:22.3326111Z GITHUB_RUN_ATTEMPT=1 2024-08-20T21:49:22.3326437Z ANACONDA_PYTHON_VERSION=3.8 2024-08-20T21:49:22.3326879Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T21:49:22.3327320Z TERM=vt100 2024-08-20T21:49:22.3327592Z INSTALLED_VISION=yes 2024-08-20T21:49:22.3327886Z BRANCH= 2024-08-20T21:49:22.3328156Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T21:49:22.3328506Z CUDA_PATH=/usr/local/cuda 2024-08-20T21:49:22.3329285Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T21:49:22.3330046Z GITHUB_SERVER_URL=https://github.com 2024-08-20T21:49:22.3330441Z UCC_COMMIT= 2024-08-20T21:49:22.3330692Z REENABLED_ISSUES= 2024-08-20T21:49:22.3330973Z DOCS=yes 2024-08-20T21:49:22.3331230Z INSTALLED_ANDROID= 2024-08-20T21:49:22.3331507Z SHLVL=1 2024-08-20T21:49:22.3331754Z MAX_JOBS=46 2024-08-20T21:49:22.3332026Z GITHUB_ACTOR_ID=54816060 2024-08-20T21:49:22.3332489Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:22.3333019Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T21:49:22.3333703Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:49:22.3334289Z GITHUB_JOB=test 2024-08-20T21:49:22.3334573Z NO_TEST_TIMEOUT=False 2024-08-20T21:49:22.3334897Z TD_DISTRIBUTED=False 2024-08-20T21:49:22.3335237Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T21:49:22.3335628Z GITHUB_RETENTION_DAYS=90 2024-08-20T21:49:22.3335954Z OPENSSL_DIR=/opt/openssl 2024-08-20T21:49:22.3336298Z GITHUB_ACTION_REPOSITORY= 2024-08-20T21:49:22.3337375Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:22.3338498Z GITHUB_BASE_REF= 2024-08-20T21:49:22.3338778Z INSTALLED_ACL= 2024-08-20T21:49:22.3339056Z CI=true 2024-08-20T21:49:22.3339341Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T21:49:22.3339714Z JOB_ID=29025551430 2024-08-20T21:49:22.3340006Z INSTALLED_PROTOBUF=yes 2024-08-20T21:49:22.3340326Z GITHUB_HEAD_REF= 2024-08-20T21:49:22.3340617Z GITHUB_ACTION_REF= 2024-08-20T21:49:22.3341051Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T21:49:22.3341524Z TEST_SHOWLOCALS=False 2024-08-20T21:49:22.3341848Z GITHUB_WORKFLOW=inductor 2024-08-20T21:49:22.3342201Z DEBIAN_FRONTEND=noninteractive 2024-08-20T21:49:22.3343124Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3343958Z NO_TD=False 2024-08-20T21:49:22.3344248Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T21:49:22.3344602Z _=/usr/bin/env 2024-08-20T21:49:22.3345069Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T21:49:22.3525956Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch 2024-08-20T21:49:22.3526884Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-20T21:49:22.3527778Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib 2024-08-20T21:49:22.3528848Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/test 2024-08-20T21:49:22.3529458Z + BUILD_DIR=build 2024-08-20T21:49:22.3529775Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T21:49:22.3530164Z + BUILD_BIN_DIR=build/bin 2024-08-20T21:49:22.3530483Z + SHARD_NUMBER=1 2024-08-20T21:49:22.3530773Z + NUM_TEST_SHARDS=2 2024-08-20T21:49:22.3531081Z + export VALGRIND=ON 2024-08-20T21:49:22.3531371Z + VALGRIND=ON 2024-08-20T21:49:22.3531787Z + [[ linux-jammy-py3.8-gcc11-build == *clang9* ]] 2024-08-20T21:49:22.3532241Z + [[ 0 == \1 ]] 2024-08-20T21:49:22.3532519Z + [[ False == \1 ]] 2024-08-20T21:49:22.3532924Z + [[ linux-jammy-py3.8-gcc11-build != *bazel* ]] 2024-08-20T21:49:22.3533411Z ++ realpath build/custom_test_artifacts 2024-08-20T21:49:22.3536391Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T21:49:22.3537087Z + [[ -n '' ]] 2024-08-20T21:49:22.3537524Z + echo 'Environment variables' 2024-08-20T21:49:22.3537897Z Environment variables 2024-08-20T21:49:22.3538208Z + env 2024-08-20T21:49:22.3542691Z INSTALLED_DB=yes 2024-08-20T21:49:22.3543226Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:49:22.3543925Z CONTINUE_THROUGH_ERROR=False 2024-08-20T21:49:22.3544404Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11-build 2024-08-20T21:49:22.3544864Z HOSTNAME=b5c5ea11556a 2024-08-20T21:49:22.3545713Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3546535Z GITHUB_ACTION=__self 2024-08-20T21:49:22.3546875Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T21:49:22.3547260Z GITHUB_RUN_NUMBER=88082 2024-08-20T21:49:22.3547673Z TEST_CONFIG=dynamic_cpu_aot_inductor_torchbench_freezing 2024-08-20T21:49:22.3548187Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T21:49:22.3548649Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T21:49:22.3549076Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T21:49:22.3549502Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T21:49:22.3549914Z GITHUB_REF_TYPE=tag 2024-08-20T21:49:22.3550225Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T21:49:22.3550645Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:22.3551073Z XLA_CUDA= 2024-08-20T21:49:22.3551541Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T21:49:22.3551923Z *** 2024-08-20T21:49:22.3552184Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T21:49:22.3552524Z GITHUB_ACTIONS=true 2024-08-20T21:49:22.3552890Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:22.3553395Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:22.3554211Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T21:49:22.3554927Z UCC_HOME=/usr 2024-08-20T21:49:22.3555207Z VERBOSE_TEST_LOGS=False 2024-08-20T21:49:22.3555570Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T21:49:22.3555998Z SHARD_NUMBER=1 2024-08-20T21:49:22.3556280Z GITHUB_REF_PROTECTED=false 2024-08-20T21:49:22.3556617Z HOME=/var/lib/jenkins 2024-08-20T21:49:22.3556982Z GITHUB_API_URL=https://api.github.com 2024-08-20T21:49:22.3557398Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T21:49:22.3557777Z UCX_COMMIT= 2024-08-20T21:49:22.3558063Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T21:49:22.3558414Z NUM_TEST_SHARDS=2 2024-08-20T21:49:22.3558686Z UCX_HOME=/usr 2024-08-20T21:49:22.3559539Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3561013Z JOB_NAME=linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:49:22.3562539Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3563686Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T21:49:22.3564497Z GITHUB_EVENT_NAME=push 2024-08-20T21:49:22.3564825Z DASHBOARD_TAG= 2024-08-20T21:49:22.3565117Z GITHUB_RUN_ID=10479310725 2024-08-20T21:49:22.3566264Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3567198Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T21:49:22.3567548Z PR_NUMBER= 2024-08-20T21:49:22.3567816Z DESIRED_CUDA= 2024-08-20T21:49:22.3568110Z GITHUB_RUN_ATTEMPT=1 2024-08-20T21:49:22.3568407Z VALGRIND=ON 2024-08-20T21:49:22.3568693Z ANACONDA_PYTHON_VERSION=3.8 2024-08-20T21:49:22.3569145Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T21:49:22.3569591Z TERM=vt100 2024-08-20T21:49:22.3569865Z INSTALLED_VISION=yes 2024-08-20T21:49:22.3570172Z BRANCH= 2024-08-20T21:49:22.3570450Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T21:49:22.3570807Z CUDA_PATH=/usr/local/cuda 2024-08-20T21:49:22.3571596Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T21:49:22.3572446Z GITHUB_SERVER_URL=https://github.com 2024-08-20T21:49:22.3572843Z UCC_COMMIT= 2024-08-20T21:49:22.3573111Z REENABLED_ISSUES= 2024-08-20T21:49:22.3573399Z DOCS=yes 2024-08-20T21:49:22.3573664Z INSTALLED_ANDROID= 2024-08-20T21:49:22.3573947Z SHLVL=1 2024-08-20T21:49:22.3574201Z MAX_JOBS=46 2024-08-20T21:49:22.3574475Z GITHUB_ACTOR_ID=54816060 2024-08-20T21:49:22.3574935Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:22.3575458Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T21:49:22.3576132Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:49:22.3576724Z GITHUB_JOB=test 2024-08-20T21:49:22.3577016Z NO_TEST_TIMEOUT=False 2024-08-20T21:49:22.3577322Z TD_DISTRIBUTED=False 2024-08-20T21:49:22.3577658Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T21:49:22.3578049Z GITHUB_RETENTION_DAYS=90 2024-08-20T21:49:22.3578373Z OPENSSL_DIR=/opt/openssl 2024-08-20T21:49:22.3578718Z GITHUB_ACTION_REPOSITORY= 2024-08-20T21:49:22.3579795Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:22.3580906Z GITHUB_BASE_REF= 2024-08-20T21:49:22.3581179Z INSTALLED_ACL= 2024-08-20T21:49:22.3581447Z CI=true 2024-08-20T21:49:22.3581722Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T21:49:22.3582087Z JOB_ID=29025551430 2024-08-20T21:49:22.3582369Z INSTALLED_PROTOBUF=yes 2024-08-20T21:49:22.3582683Z GITHUB_HEAD_REF= 2024-08-20T21:49:22.3582968Z GITHUB_ACTION_REF= 2024-08-20T21:49:22.3583385Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T21:49:22.3583853Z TEST_SHOWLOCALS=False 2024-08-20T21:49:22.3584171Z GITHUB_WORKFLOW=inductor 2024-08-20T21:49:22.3584519Z DEBIAN_FRONTEND=noninteractive 2024-08-20T21:49:22.3585445Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_0135416e-dd97-4541-968a-e03172c296e9 2024-08-20T21:49:22.3586273Z NO_TD=False 2024-08-20T21:49:22.3586559Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T21:49:22.3586921Z _=/usr/bin/env 2024-08-20T21:49:22.3587231Z + echo 'Testing pytorch' 2024-08-20T21:49:22.3587559Z Testing pytorch 2024-08-20T21:49:22.3587866Z + export LANG=C.UTF-8 2024-08-20T21:49:22.3588173Z + LANG=C.UTF-8 2024-08-20T21:49:22.3588449Z + PR_NUMBER= 2024-08-20T21:49:22.3588890Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == \d\e\f\a\u\l\t ]] 2024-08-20T21:49:22.3589628Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T21:49:22.3590326Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == \s\l\o\w ]] 2024-08-20T21:49:22.3591019Z + [[ linux-jammy-py3.8-gcc11-build == *slow-gradcheck* ]] 2024-08-20T21:49:22.3591628Z + [[ linux-jammy-py3.8-gcc11-build == *cuda* ]] 2024-08-20T21:49:22.3592186Z + [[ linux-jammy-py3.8-gcc11-build == *rocm* ]] 2024-08-20T21:49:22.3592725Z + [[ linux-jammy-py3.8-gcc11-build == *xpu* ]] 2024-08-20T21:49:22.3593306Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *crossref* ]] 2024-08-20T21:49:22.3593949Z + [[ linux-jammy-py3.8-gcc11-build == *rocm* ]] 2024-08-20T21:49:22.3594573Z + [[ linux-jammy-py3.8-gcc11-build == *xpu* ]] 2024-08-20T21:49:22.3595132Z + [[ linux-jammy-py3.8-gcc11-build != *-bazel-* ]] 2024-08-20T21:49:22.3595661Z + pip_install --user ninja==1.10.2 2024-08-20T21:49:22.3596196Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T21:49:22.7583953Z Collecting ninja==1.10.2 2024-08-20T21:49:22.7705843Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T21:49:22.7804658Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T21:49:23.4193387Z Installing collected packages: ninja 2024-08-20T21:49:23.4271335Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T21:49:23.4272680Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T21:49:23.4323210Z Successfully installed ninja-1.10.2 2024-08-20T21:49:23.4926179Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:23.4928425Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:23.4929985Z + [[ linux-jammy-py3.8-gcc11-build == *aarch64* ]] 2024-08-20T21:49:23.4930444Z + install_tlparse 2024-08-20T21:49:23.4930834Z + pip_install --user tlparse==0.3.25 2024-08-20T21:49:23.4931393Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T21:49:23.9289869Z Collecting tlparse==0.3.25 2024-08-20T21:49:23.9396311Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T21:49:23.9799788Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T21:49:24.6605207Z Installing collected packages: tlparse 2024-08-20T21:49:24.8354655Z Successfully installed tlparse-0.3.25 2024-08-20T21:49:24.9119183Z ++ python -m site --user-base 2024-08-20T21:49:24.9391462Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:24.9393541Z + [[ linux-jammy-py3.8-gcc11-build == *asan* ]] 2024-08-20T21:49:24.9394139Z + [[ linux-jammy-py3.8-gcc11-build == *-debug* ]] 2024-08-20T21:49:24.9394722Z + [[ linux-jammy-py3.8-gcc11-build != *-bazel-* ]] 2024-08-20T21:49:24.9395585Z + echo 'We are not in debug mode: linux-jammy-py3.8-gcc11-build. Expect the assertion to pass' 2024-08-20T21:49:24.9396662Z We are not in debug mode: linux-jammy-py3.8-gcc11-build. Expect the assertion to pass 2024-08-20T21:49:24.9397331Z + cd test 2024-08-20T21:49:24.9397854Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T21:49:26.2931231Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T21:49:26.2932265Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T21:49:26.2933382Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T21:49:26.2933928Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *pr_time_benchmarks* ]] 2024-08-20T21:49:26.2934670Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *dynamo_eager* ]] 2024-08-20T21:49:26.2935353Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *aot_eager* ]] 2024-08-20T21:49:26.2936026Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *aot_inductor* ]] 2024-08-20T21:49:26.2936874Z + DYNAMO_BENCHMARK_FLAGS+=(--export-aot-inductor) 2024-08-20T21:49:26.2937464Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *dynamic* ]] 2024-08-20T21:49:26.2938213Z + DYNAMO_BENCHMARK_FLAGS+=(--dynamic-shapes --dynamic-batch-only) 2024-08-20T21:49:26.2939119Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *cpu* ]] 2024-08-20T21:49:26.2939717Z + DYNAMO_BENCHMARK_FLAGS+=(--device cpu) 2024-08-20T21:49:26.2956407Z + [[ linux-jammy-py3.8-gcc11-build == *libtorch* ]] 2024-08-20T21:49:26.2957043Z + [[ linux-jammy-py3.8-gcc11-build == *-bazel-* ]] 2024-08-20T21:49:26.2957481Z + cd test 2024-08-20T21:49:26.2957921Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T21:49:27.5623846Z PyTorch built with: 2024-08-20T21:49:27.5624439Z - GCC 11.4 2024-08-20T21:49:27.5624778Z - C++ Version: 201703 2024-08-20T21:49:27.5625656Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T21:49:27.5626775Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T21:49:27.5627448Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T21:49:27.5628308Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T21:49:27.5628773Z - NNPACK is enabled 2024-08-20T21:49:27.5629157Z - CPU capability usage: AVX512 2024-08-20T21:49:27.5636893Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T21:49:27.5643658Z 2024-08-20T21:49:27.7768060Z + cd test 2024-08-20T21:49:28.9205552Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T21:49:28.9206132Z ATen/Parallel: 2024-08-20T21:49:28.9206460Z at::get_num_threads() : 24 2024-08-20T21:49:28.9206828Z at::get_num_interop_threads() : 24 2024-08-20T21:49:28.9207254Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T21:49:28.9207632Z omp_get_max_threads() : 24 2024-08-20T21:49:28.9208551Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T21:49:28.9209402Z mkl_get_max_threads() : 24 2024-08-20T21:49:28.9210027Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T21:49:28.9210671Z std::thread::hardware_concurrency() : 48 2024-08-20T21:49:28.9211096Z Environment variables: 2024-08-20T21:49:28.9211430Z OMP_NUM_THREADS : [not set] 2024-08-20T21:49:28.9211771Z MKL_NUM_THREADS : [not set] 2024-08-20T21:49:28.9212127Z ATen parallel backend: OpenMP 2024-08-20T21:49:28.9212386Z 2024-08-20T21:49:29.1038708Z + [[ linux-jammy-py3.8-gcc11-build == *aarch64* ]] 2024-08-20T21:49:29.1039420Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *backward* ]] 2024-08-20T21:49:29.1040063Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *xla* ]] 2024-08-20T21:49:29.1040734Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *executorch* ]] 2024-08-20T21:49:29.1041459Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T21:49:29.1042308Z + [[ linux-jammy-py3.8-gcc11-build == *libtorch* ]] 2024-08-20T21:49:29.1042940Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == distributed ]] 2024-08-20T21:49:29.1043962Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *inductor_distributed* ]] 2024-08-20T21:49:29.1044823Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *inductor-halide* ]] 2024-08-20T21:49:29.1045696Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *inductor-micro-benchmark* ]] 2024-08-20T21:49:29.1046471Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *huggingface* ]] 2024-08-20T21:49:29.1047116Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *timm* ]] 2024-08-20T21:49:29.1047771Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *torchbench* ]] 2024-08-20T21:49:29.1048422Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *cpu* ]] 2024-08-20T21:49:29.1048928Z + install_torchaudio cpu 2024-08-20T21:49:29.1049240Z + local commit 2024-08-20T21:49:29.1049529Z ++ get_pinned_commit audio 2024-08-20T21:49:29.1049971Z ++ cat .github/ci_commit_pins/audio.txt 2024-08-20T21:49:29.1050679Z + commit=b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:29.1051117Z + [[ cpu == \c\u\d\a ]] 2024-08-20T21:49:29.1052042Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:29.1053572Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:29.4253228Z Collecting git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:29.4257451Z Cloning https://github.com/pytorch/audio.git (to revision b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862) to /tmp/pip-req-build-vb8f9o_h 2024-08-20T21:49:29.4290208Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio.git /tmp/pip-req-build-vb8f9o_h 2024-08-20T21:49:30.1675777Z Running command git rev-parse -q --verify 'sha^b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862' 2024-08-20T21:49:30.1711788Z Running command git fetch -q https://github.com/pytorch/audio.git b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:30.2844581Z Resolved https://github.com/pytorch/audio.git to commit b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:30.2845636Z Running command git submodule update --init --recursive -q 2024-08-20T21:49:32.2135051Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:49:32.2183440Z [?25hRequirement already satisfied: torch in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchaudio==2.4.0a0+b3f6f51) (2.5.0a0+git40ec5f6) 2024-08-20T21:49:32.2212679Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.13.1) 2024-08-20T21:49:32.2214523Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (4.12.2) 2024-08-20T21:49:32.2216813Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2.8.8) 2024-08-20T21:49:32.2219501Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.1.4) 2024-08-20T21:49:32.2222467Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2024.6.1) 2024-08-20T21:49:32.2228325Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (1.12.1) 2024-08-20T21:49:32.2243988Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch->torchaudio==2.4.0a0+b3f6f51) (1.3.0) 2024-08-20T21:49:32.2604134Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch->torchaudio==2.4.0a0+b3f6f51) (2.1.5) 2024-08-20T21:49:32.2665404Z Building wheels for collected packages: torchaudio 2024-08-20T21:50:26.5469879Z Building wheel for torchaudio (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-08-20T21:50:26.5525074Z [?25h Created wheel for torchaudio: filename=torchaudio-2.4.0a0+b3f6f51-cp38-cp38-linux_x86_64.whl size=1767370 sha256=06b90a29c76e3ec2d309746ae0ef220fb5d64db11a45ef297809cfcf4346ef36 2024-08-20T21:50:26.5526811Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/2d/7d/0d/bdfdb3f2b184f238c1d61b3f9b248992c036dfdb76e5397d45 2024-08-20T21:50:26.5560835Z Successfully built torchaudio 2024-08-20T21:50:27.1325245Z Installing collected packages: torchaudio 2024-08-20T21:50:27.3263653Z Successfully installed torchaudio-2.4.0a0+b3f6f51 2024-08-20T21:50:27.4405884Z + install_torchtext 2024-08-20T21:50:27.4406260Z + local data_commit 2024-08-20T21:50:27.4406557Z + local text_commit 2024-08-20T21:50:27.4408460Z ++ get_pinned_commit data 2024-08-20T21:50:27.4408884Z ++ cat .github/ci_commit_pins/data.txt 2024-08-20T21:50:27.4418392Z + data_commit=11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:27.4422391Z ++ get_pinned_commit text 2024-08-20T21:50:27.4422977Z ++ cat .github/ci_commit_pins/text.txt 2024-08-20T21:50:27.4432353Z + text_commit=b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:50:27.4433608Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:27.4435134Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:27.7638375Z Collecting git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:27.7639980Z Cloning https://github.com/pytorch/data.git (to revision 11bb5b847ea8b9e0d9bb82db3304daf383008d3f) to /tmp/pip-req-build-1k347p3u 2024-08-20T21:50:27.7671309Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/data.git /tmp/pip-req-build-1k347p3u 2024-08-20T21:50:28.1828964Z Running command git rev-parse -q --verify 'sha^11bb5b847ea8b9e0d9bb82db3304daf383008d3f' 2024-08-20T21:50:28.1864434Z Running command git fetch -q https://github.com/pytorch/data.git 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:28.2721052Z Running command git checkout -q 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:28.4405755Z Resolved https://github.com/pytorch/data.git to commit 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:28.4407015Z Running command git submodule update --init --recursive -q 2024-08-20T21:51:16.1161538Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:51:16.1207665Z [?25hRequirement already satisfied: urllib3>=1.25 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchdata==0.7.0a0+11bb5b8) (1.26.19) 2024-08-20T21:51:16.1210303Z Requirement already satisfied: requests in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.32.3) 2024-08-20T21:51:16.1214646Z Requirement already satisfied: torch>2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.5.0a0+git40ec5f6) 2024-08-20T21:51:16.1246726Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.13.1) 2024-08-20T21:51:16.1250737Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (4.12.2) 2024-08-20T21:51:16.1253892Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.8.8) 2024-08-20T21:51:16.1257067Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.1.4) 2024-08-20T21:51:16.1260465Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2024.6.1) 2024-08-20T21:51:16.1265951Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.12.1) 2024-08-20T21:51:16.1281458Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.3.0) 2024-08-20T21:51:16.1341886Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.3.2) 2024-08-20T21:51:16.1344967Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.7) 2024-08-20T21:51:16.1347480Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (2024.7.4) 2024-08-20T21:51:16.1733760Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.1.5) 2024-08-20T21:51:16.1803837Z Building wheels for collected packages: torchdata 2024-08-20T21:51:17.8293365Z Building wheel for torchdata (setup.py) ... [?25l- \ | done 2024-08-20T21:51:17.8310690Z [?25h Created wheel for torchdata: filename=torchdata-0.7.0a0+11bb5b8-py3-none-any.whl size=182970 sha256=0bf4b536be9665527e4f2ca6cdb54fd7ec6ead87335e19799e2fd934aeebf604 2024-08-20T21:51:17.8312371Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/f2/b8/b4/f1929a760cf6ea9bda553072159f9dc096899c37ad8a5ffafe 2024-08-20T21:51:17.8347002Z Successfully built torchdata 2024-08-20T21:51:18.3923897Z Installing collected packages: torchdata 2024-08-20T21:51:18.4810207Z Successfully installed torchdata-0.7.0a0+11bb5b8 2024-08-20T21:51:20.5286530Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:20.5288372Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:20.8462888Z Collecting git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:20.8464797Z Cloning https://github.com/pytorch/text.git (to revision b0ebddc648d279826089db91775375221777a2db) to /tmp/pip-req-build-o4ahz75v 2024-08-20T21:51:20.8494789Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/text.git /tmp/pip-req-build-o4ahz75v 2024-08-20T21:51:21.8061697Z Running command git rev-parse -q --verify 'sha^b0ebddc648d279826089db91775375221777a2db' 2024-08-20T21:51:21.8099945Z Running command git fetch -q https://github.com/pytorch/text.git b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:21.8985861Z Running command git checkout -q b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:22.0538441Z Resolved https://github.com/pytorch/text.git to commit b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:22.0539571Z Running command git submodule update --init --recursive -q 2024-08-20T21:51:28.3846536Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:51:28.3894596Z [?25hRequirement already satisfied: tqdm in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (4.66.5) 2024-08-20T21:51:28.3897180Z Requirement already satisfied: requests in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.32.3) 2024-08-20T21:51:28.3900590Z Requirement already satisfied: torch in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.5.0a0+git40ec5f6) 2024-08-20T21:51:28.3903685Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (1.21.2) 2024-08-20T21:51:28.3907269Z Requirement already satisfied: torchdata in /var/lib/jenkins/.local/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (0.7.0a0+11bb5b8) 2024-08-20T21:51:28.3939172Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.3.2) 2024-08-20T21:51:28.3941020Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.7) 2024-08-20T21:51:28.3943916Z Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (1.26.19) 2024-08-20T21:51:28.3947852Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (2024.7.4) 2024-08-20T21:51:28.3977629Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.13.1) 2024-08-20T21:51:28.3979965Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (4.12.2) 2024-08-20T21:51:28.3982742Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2.8.8) 2024-08-20T21:51:28.3985888Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.1.4) 2024-08-20T21:51:28.3989598Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2024.6.1) 2024-08-20T21:51:28.3995067Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (1.12.1) 2024-08-20T21:51:28.4007915Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch->torchtext==0.16.0a0+b0ebddc) (1.3.0) 2024-08-20T21:51:28.4478294Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch->torchtext==0.16.0a0+b0ebddc) (2.1.5) 2024-08-20T21:51:28.4665615Z Building wheels for collected packages: torchtext 2024-08-20T21:51:56.8612990Z Building wheel for torchtext (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-08-20T21:51:56.8679622Z [?25h Created wheel for torchtext: filename=torchtext-0.16.0a0+b0ebddc-cp38-cp38-linux_x86_64.whl size=2045013 sha256=4641ec0179342d1d23cbba576e783d8f39365a5912768fc582c9381aff4c7caf 2024-08-20T21:51:56.8681457Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/d6/1b/67/0243915f173a4bc67968d5441b501e3df9c77a50521995f939 2024-08-20T21:51:56.8713636Z Successfully built torchtext 2024-08-20T21:51:57.4259479Z Installing collected packages: torchtext 2024-08-20T21:51:57.5332879Z Successfully installed torchtext-0.16.0a0+b0ebddc 2024-08-20T21:51:57.6404303Z + install_torchvision 2024-08-20T21:51:57.6404728Z + local orig_preload 2024-08-20T21:51:57.6405866Z + local commit 2024-08-20T21:51:57.6406724Z ++ get_pinned_commit vision 2024-08-20T21:51:57.6407122Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T21:51:57.6415820Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:57.6416328Z + orig_preload= 2024-08-20T21:51:57.6416786Z + '[' -n '' ']' 2024-08-20T21:51:57.6417651Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:57.6419175Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:57.9626861Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:57.9628528Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-qavy9dve 2024-08-20T21:51:57.9661387Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-qavy9dve 2024-08-20T21:51:59.3016894Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T21:51:59.3053384Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:59.4020045Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:59.6683080Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:52:01.6981069Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:52:01.7028689Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-08-20T21:52:01.7030432Z Requirement already satisfied: torch in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T21:52:01.7033267Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T21:52:01.7108396Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T21:52:01.7110239Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T21:52:01.7112108Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T21:52:01.7115310Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T21:52:01.7118384Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T21:52:01.7124051Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1) 2024-08-20T21:52:01.7140286Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T21:52:01.7501803Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T21:52:01.7563461Z Building wheels for collected packages: torchvision 2024-08-20T21:52:27.0948501Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-08-20T21:52:27.0990956Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp38-cp38-linux_x86_64.whl size=1215628 sha256=c2a0213bd7fc8fe9fe87fb71aa69bf1f2d9dfca59203f72d92c42e09792d85c5 2024-08-20T21:52:27.0992716Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/82/f9/1e/f126983e23cfd567a4cb268f7d0778919a2a7014fe9af04f11 2024-08-20T21:52:27.1162991Z Successfully built torchvision 2024-08-20T21:52:27.6772934Z Installing collected packages: torchvision 2024-08-20T21:52:28.0564599Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T21:52:28.1463335Z + '[' -n '' ']' 2024-08-20T21:52:28.1464013Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-20T21:52:28.1465989Z + pip_install git+https://github.com/pytorch/ao.git 2024-08-20T21:52:28.1466889Z + pip install --progress-bar off git+https://github.com/pytorch/ao.git 2024-08-20T21:52:28.4660589Z Collecting git+https://github.com/pytorch/ao.git 2024-08-20T21:52:28.4661700Z Cloning https://github.com/pytorch/ao.git to /tmp/pip-req-build-ya36bz_r 2024-08-20T21:52:28.4691362Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/ao.git /tmp/pip-req-build-ya36bz_r 2024-08-20T21:52:29.0339341Z Resolved https://github.com/pytorch/ao.git to commit 2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T21:52:30.9660224Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:52:30.9709257Z [?25hBuilding wheels for collected packages: torchao 2024-08-20T21:52:44.8346078Z Building wheel for torchao (setup.py) ... [?25l- \ | / - \ done 2024-08-20T21:52:44.8445637Z [?25h Created wheel for torchao: filename=torchao-0.4.0+git2c8e3f3-cp38-cp38-linux_x86_64.whl size=3735231 sha256=3ed8d8d0f370feaa72ddcf0ae07ebe6eed64c531c56d66f6dd308a061f07e6f1 2024-08-20T21:52:44.8447890Z Stored in directory: /tmp/pip-ephem-wheel-cache-ingiuar2/wheels/7a/d0/ee/d845e81513be33ad662e10ba9fb85a811ff1ad92ea52f8b6a9 2024-08-20T21:52:44.8485131Z Successfully built torchao 2024-08-20T21:52:45.4715623Z Installing collected packages: torchao 2024-08-20T21:52:45.7364755Z Successfully installed torchao-0.4.0+git2c8e3f3 2024-08-20T21:52:45.8127795Z + id=0 2024-08-20T21:52:45.8128368Z + pip_install opencv-python==4.8.0.74 2024-08-20T21:52:45.8128985Z + pip install --progress-bar off opencv-python==4.8.0.74 2024-08-20T21:52:46.2443962Z Collecting opencv-python==4.8.0.74 2024-08-20T21:52:46.2552632Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB) 2024-08-20T21:52:46.2630991Z Requirement already satisfied: numpy>=1.17.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opencv-python==4.8.0.74) (1.21.2) 2024-08-20T21:52:46.2670380Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB) 2024-08-20T21:52:47.3081515Z Installing collected packages: opencv-python 2024-08-20T21:52:48.1628409Z Successfully installed opencv-python-4.8.0.74 2024-08-20T21:52:48.2393141Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *inductor_torchbench_smoketest_perf* ]] 2024-08-20T21:52:48.2394153Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *inductor_torchbench_cpu_smoketest_perf* ]] 2024-08-20T21:52:48.2395071Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *torchbench_gcp_smoketest* ]] 2024-08-20T21:52:48.2395705Z + checkout_install_torchbench 2024-08-20T21:52:48.2396049Z + local commit 2024-08-20T21:52:48.2396347Z ++ get_pinned_commit torchbench 2024-08-20T21:52:48.2396763Z ++ cat .github/ci_commit_pins/torchbench.txt 2024-08-20T21:52:48.2404483Z + commit=23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T21:52:48.2405069Z + git clone https://github.com/pytorch/benchmark torchbench 2024-08-20T21:52:48.2422034Z Cloning into 'torchbench'... 2024-08-20T21:52:48.3951152Z remote: Enumerating objects: 30997, done. 2024-08-20T21:52:48.3952619Z remote: Counting objects: 0% (1/1737) 2024-08-20T21:52:48.3953156Z remote: Counting objects: 1% (18/1737) 2024-08-20T21:52:48.3954705Z remote: Counting objects: 2% (35/1737) 2024-08-20T21:52:48.3955311Z remote: Counting objects: 3% (53/1737) 2024-08-20T21:52:48.3955922Z remote: Counting objects: 4% (70/1737) 2024-08-20T21:52:48.3956430Z remote: Counting objects: 5% (87/1737) 2024-08-20T21:52:48.3957014Z remote: Counting objects: 6% (105/1737) 2024-08-20T21:52:48.3957790Z remote: Counting objects: 7% (122/1737) 2024-08-20T21:52:48.3958378Z remote: Counting objects: 8% (139/1737) 2024-08-20T21:52:48.3958948Z remote: Counting objects: 9% (157/1737) 2024-08-20T21:52:48.3959474Z remote: Counting objects: 10% (174/1737) 2024-08-20T21:52:48.3960066Z remote: Counting objects: 11% (192/1737) 2024-08-20T21:52:48.3960647Z remote: Counting objects: 12% (209/1737) 2024-08-20T21:52:48.3962335Z remote: Counting objects: 13% (226/1737) 2024-08-20T21:52:48.3962922Z remote: Counting objects: 14% (244/1737) 2024-08-20T21:52:48.3965297Z remote: Counting objects: 15% (261/1737) 2024-08-20T21:52:48.3966035Z remote: Counting objects: 16% (278/1737) 2024-08-20T21:52:48.3966607Z remote: Counting objects: 17% (296/1737) 2024-08-20T21:52:48.3967193Z remote: Counting objects: 18% (313/1737) 2024-08-20T21:52:48.3967721Z remote: Counting objects: 19% (331/1737) 2024-08-20T21:52:48.3968306Z remote: Counting objects: 20% (348/1737) 2024-08-20T21:52:48.3968877Z remote: Counting objects: 21% (365/1737) 2024-08-20T21:52:48.3969453Z remote: Counting objects: 22% (383/1737) 2024-08-20T21:52:48.3970023Z remote: Counting objects: 23% (400/1737) 2024-08-20T21:52:48.3970595Z remote: Counting objects: 24% (417/1737) 2024-08-20T21:52:48.3971176Z remote: Counting objects: 25% (435/1737) 2024-08-20T21:52:48.3971735Z remote: Counting objects: 26% (452/1737) 2024-08-20T21:52:48.3972483Z remote: Counting objects: 27% (469/1737) 2024-08-20T21:52:48.3973033Z remote: Counting objects: 28% (487/1737) 2024-08-20T21:52:48.3973616Z remote: Counting objects: 29% (504/1737) 2024-08-20T21:52:48.3974179Z remote: Counting objects: 30% (522/1737) 2024-08-20T21:52:48.3974749Z remote: Counting objects: 31% (539/1737) 2024-08-20T21:52:48.3975266Z remote: Counting objects: 32% (556/1737) 2024-08-20T21:52:48.3975837Z remote: Counting objects: 33% (574/1737) 2024-08-20T21:52:48.3976411Z remote: Counting objects: 34% (591/1737) 2024-08-20T21:52:48.3976977Z remote: Counting objects: 35% (608/1737) 2024-08-20T21:52:48.3977498Z remote: Counting objects: 36% (626/1737) 2024-08-20T21:52:48.3978062Z remote: Counting objects: 37% (643/1737) 2024-08-20T21:52:48.3978636Z remote: Counting objects: 38% (661/1737) 2024-08-20T21:52:48.3979185Z remote: Counting objects: 39% (678/1737) 2024-08-20T21:52:48.3979734Z remote: Counting objects: 40% (695/1737) 2024-08-20T21:52:48.3980297Z remote: Counting objects: 41% (713/1737) 2024-08-20T21:52:48.3980881Z remote: Counting objects: 42% (730/1737) 2024-08-20T21:52:48.3981406Z remote: Counting objects: 43% (747/1737) 2024-08-20T21:52:48.3981973Z remote: Counting objects: 44% (765/1737) 2024-08-20T21:52:48.3982540Z remote: Counting objects: 45% (782/1737) 2024-08-20T21:52:48.3983110Z remote: Counting objects: 46% (800/1737) 2024-08-20T21:52:48.3983629Z remote: Counting objects: 47% (817/1737) 2024-08-20T21:52:48.3984208Z remote: Counting objects: 48% (834/1737) 2024-08-20T21:52:48.3984765Z remote: Counting objects: 49% (852/1737) 2024-08-20T21:52:48.3985342Z remote: Counting objects: 50% (869/1737) 2024-08-20T21:52:48.3985864Z remote: Counting objects: 51% (886/1737) 2024-08-20T21:52:48.3986428Z remote: Counting objects: 52% (904/1737) 2024-08-20T21:52:48.3986996Z remote: Counting objects: 53% (921/1737) 2024-08-20T21:52:48.3987571Z remote: Counting objects: 54% (938/1737) 2024-08-20T21:52:48.3988094Z remote: Counting objects: 55% (956/1737) 2024-08-20T21:52:48.3988668Z remote: Counting objects: 56% (973/1737) 2024-08-20T21:52:48.3989243Z remote: Counting objects: 57% (991/1737) 2024-08-20T21:52:48.3989767Z remote: Counting objects: 58% (1008/1737) 2024-08-20T21:52:48.3990362Z remote: Counting objects: 59% (1025/1737) 2024-08-20T21:52:48.3990877Z remote: Counting objects: 60% (1043/1737) 2024-08-20T21:52:48.3991462Z remote: Counting objects: 61% (1060/1737) 2024-08-20T21:52:48.3991989Z remote: Counting objects: 62% (1077/1737) 2024-08-20T21:52:48.3992513Z remote: Counting objects: 63% (1095/1737) 2024-08-20T21:52:48.3993025Z remote: Counting objects: 64% (1112/1737) 2024-08-20T21:52:48.3993599Z remote: Counting objects: 65% (1130/1737) 2024-08-20T21:52:48.3994127Z remote: Counting objects: 66% (1147/1737) 2024-08-20T21:52:48.3994650Z remote: Counting objects: 67% (1164/1737) 2024-08-20T21:52:48.3995167Z remote: Counting objects: 68% (1182/1737) 2024-08-20T21:52:48.3995686Z remote: Counting objects: 69% (1199/1737) 2024-08-20T21:52:48.3996370Z remote: Counting objects: 70% (1216/1737) 2024-08-20T21:52:48.3996902Z remote: Counting objects: 71% (1234/1737) 2024-08-20T21:52:48.3997470Z remote: Counting objects: 72% (1251/1737) 2024-08-20T21:52:48.3997993Z remote: Counting objects: 73% (1269/1737) 2024-08-20T21:52:48.3998515Z remote: Counting objects: 74% (1286/1737) 2024-08-20T21:52:48.3999019Z remote: Counting objects: 75% (1303/1737) 2024-08-20T21:52:48.3999546Z remote: Counting objects: 76% (1321/1737) 2024-08-20T21:52:48.4000066Z remote: Counting objects: 77% (1338/1737) 2024-08-20T21:52:48.4000581Z remote: Counting objects: 78% (1355/1737) 2024-08-20T21:52:48.4001088Z remote: Counting objects: 79% (1373/1737) 2024-08-20T21:52:48.4001689Z remote: Counting objects: 80% (1390/1737) 2024-08-20T21:52:48.4002211Z remote: Counting objects: 81% (1407/1737) 2024-08-20T21:52:48.4002827Z remote: Counting objects: 82% (1425/1737) 2024-08-20T21:52:48.4003337Z remote: Counting objects: 83% (1442/1737) 2024-08-20T21:52:48.4003861Z remote: Counting objects: 84% (1460/1737) 2024-08-20T21:52:48.4004450Z remote: Counting objects: 85% (1477/1737) 2024-08-20T21:52:48.4004974Z remote: Counting objects: 86% (1494/1737) 2024-08-20T21:52:48.4005481Z remote: Counting objects: 87% (1512/1737) 2024-08-20T21:52:48.4006002Z remote: Counting objects: 88% (1529/1737) 2024-08-20T21:52:48.4006517Z remote: Counting objects: 89% (1546/1737) 2024-08-20T21:52:48.4007057Z remote: Counting objects: 90% (1564/1737) 2024-08-20T21:52:48.4007580Z remote: Counting objects: 91% (1581/1737) 2024-08-20T21:52:48.4008084Z remote: Counting objects: 92% (1599/1737) 2024-08-20T21:52:48.4008605Z remote: Counting objects: 93% (1616/1737) 2024-08-20T21:52:48.4009126Z remote: Counting objects: 94% (1633/1737) 2024-08-20T21:52:48.4009636Z remote: Counting objects: 95% (1651/1737) 2024-08-20T21:52:48.4010157Z remote: Counting objects: 96% (1668/1737) 2024-08-20T21:52:48.4010681Z remote: Counting objects: 97% (1685/1737) 2024-08-20T21:52:48.4011201Z remote: Counting objects: 98% (1703/1737) 2024-08-20T21:52:48.4011705Z remote: Counting objects: 99% (1720/1737) 2024-08-20T21:52:48.4012222Z remote: Counting objects: 100% (1737/1737) 2024-08-20T21:52:48.4012783Z remote: Counting objects: 100% (1737/1737), done. 2024-08-20T21:52:48.4013355Z remote: Compressing objects: 0% (1/898) 2024-08-20T21:52:48.4065687Z remote: Compressing objects: 1% (9/898) 2024-08-20T21:52:48.4182557Z remote: Compressing objects: 2% (18/898) 2024-08-20T21:52:48.4261804Z remote: Compressing objects: 3% (27/898) 2024-08-20T21:52:48.4327689Z remote: Compressing objects: 4% (36/898) 2024-08-20T21:52:48.4366992Z remote: Compressing objects: 5% (45/898) 2024-08-20T21:52:48.4397744Z remote: Compressing objects: 6% (54/898) 2024-08-20T21:52:48.4427228Z remote: Compressing objects: 7% (63/898) 2024-08-20T21:52:48.4456905Z remote: Compressing objects: 8% (72/898) 2024-08-20T21:52:48.4486479Z remote: Compressing objects: 9% (81/898) 2024-08-20T21:52:48.4522286Z remote: Compressing objects: 10% (90/898) 2024-08-20T21:52:48.4531468Z remote: Compressing objects: 11% (99/898) 2024-08-20T21:52:48.4546842Z remote: Compressing objects: 12% (108/898) 2024-08-20T21:52:48.4560953Z remote: Compressing objects: 13% (117/898) 2024-08-20T21:52:48.4583596Z remote: Compressing objects: 14% (126/898) 2024-08-20T21:52:48.4585267Z remote: Compressing objects: 15% (135/898) 2024-08-20T21:52:48.4597597Z remote: Compressing objects: 16% (144/898) 2024-08-20T21:52:48.4599269Z remote: Compressing objects: 17% (153/898) 2024-08-20T21:52:48.4601656Z remote: Compressing objects: 18% (162/898) 2024-08-20T21:52:48.4603774Z remote: Compressing objects: 19% (171/898) 2024-08-20T21:52:48.4605373Z remote: Compressing objects: 20% (180/898) 2024-08-20T21:52:48.4609753Z remote: Compressing objects: 21% (189/898) 2024-08-20T21:52:48.4611725Z remote: Compressing objects: 22% (198/898) 2024-08-20T21:52:48.4613620Z remote: Compressing objects: 23% (207/898) 2024-08-20T21:52:48.4615054Z remote: Compressing objects: 24% (216/898) 2024-08-20T21:52:48.4615586Z remote: Compressing objects: 25% (225/898) 2024-08-20T21:52:48.4618926Z remote: Compressing objects: 26% (234/898) 2024-08-20T21:52:48.4627224Z remote: Compressing objects: 27% (243/898) 2024-08-20T21:52:48.4630669Z remote: Compressing objects: 28% (252/898) 2024-08-20T21:52:48.4633818Z remote: Compressing objects: 29% (261/898) 2024-08-20T21:52:48.4636461Z remote: Compressing objects: 30% (270/898) 2024-08-20T21:52:48.4639616Z remote: Compressing objects: 31% (279/898) 2024-08-20T21:52:48.4642402Z remote: Compressing objects: 32% (288/898) 2024-08-20T21:52:48.4646303Z remote: Compressing objects: 33% (297/898) 2024-08-20T21:52:48.4649404Z remote: Compressing objects: 34% (306/898) 2024-08-20T21:52:48.4654320Z remote: Compressing objects: 35% (315/898) 2024-08-20T21:52:48.4657379Z remote: Compressing objects: 36% (324/898) 2024-08-20T21:52:48.4660005Z remote: Compressing objects: 37% (333/898) 2024-08-20T21:52:48.4661975Z remote: Compressing objects: 38% (342/898) 2024-08-20T21:52:48.4662625Z remote: Compressing objects: 39% (351/898) 2024-08-20T21:52:48.4665465Z remote: Compressing objects: 40% (360/898) 2024-08-20T21:52:48.4666016Z remote: Compressing objects: 41% (369/898) 2024-08-20T21:52:48.4666546Z remote: Compressing objects: 42% (378/898) 2024-08-20T21:52:48.4668391Z remote: Compressing objects: 43% (387/898) 2024-08-20T21:52:48.4671565Z remote: Compressing objects: 44% (396/898) 2024-08-20T21:52:48.4672110Z remote: Compressing objects: 45% (405/898) 2024-08-20T21:52:48.4672661Z remote: Compressing objects: 46% (414/898) 2024-08-20T21:52:48.4673191Z remote: Compressing objects: 47% (423/898) 2024-08-20T21:52:48.4673728Z remote: Compressing objects: 48% (432/898) 2024-08-20T21:52:48.4674266Z remote: Compressing objects: 49% (441/898) 2024-08-20T21:52:48.4674799Z remote: Compressing objects: 50% (449/898) 2024-08-20T21:52:48.4675322Z remote: Compressing objects: 51% (458/898) 2024-08-20T21:52:48.4675854Z remote: Compressing objects: 52% (467/898) 2024-08-20T21:52:48.4676383Z remote: Compressing objects: 53% (476/898) 2024-08-20T21:52:48.4676900Z remote: Compressing objects: 54% (485/898) 2024-08-20T21:52:48.4677440Z remote: Compressing objects: 55% (494/898) 2024-08-20T21:52:48.4677973Z remote: Compressing objects: 56% (503/898) 2024-08-20T21:52:48.4678504Z remote: Compressing objects: 57% (512/898) 2024-08-20T21:52:48.4679021Z remote: Compressing objects: 58% (521/898) 2024-08-20T21:52:48.4679552Z remote: Compressing objects: 59% (530/898) 2024-08-20T21:52:48.4680093Z remote: Compressing objects: 60% (539/898) 2024-08-20T21:52:48.4680628Z remote: Compressing objects: 61% (548/898) 2024-08-20T21:52:48.4681147Z remote: Compressing objects: 62% (557/898) 2024-08-20T21:52:48.4681791Z remote: Compressing objects: 63% (566/898) 2024-08-20T21:52:48.4682339Z remote: Compressing objects: 64% (575/898) 2024-08-20T21:52:48.4682874Z remote: Compressing objects: 65% (584/898) 2024-08-20T21:52:48.4683396Z remote: Compressing objects: 66% (593/898) 2024-08-20T21:52:48.4683924Z remote: Compressing objects: 67% (602/898) 2024-08-20T21:52:48.4684457Z remote: Compressing objects: 68% (611/898) 2024-08-20T21:52:48.4684989Z remote: Compressing objects: 69% (620/898) 2024-08-20T21:52:48.4685506Z remote: Compressing objects: 70% (629/898) 2024-08-20T21:52:48.4686039Z remote: Compressing objects: 71% (638/898) 2024-08-20T21:52:48.4686567Z remote: Compressing objects: 72% (647/898) 2024-08-20T21:52:48.4687096Z remote: Compressing objects: 73% (656/898) 2024-08-20T21:52:48.4687618Z remote: Compressing objects: 74% (665/898) 2024-08-20T21:52:48.4688159Z remote: Compressing objects: 75% (674/898) 2024-08-20T21:52:48.4704606Z remote: Compressing objects: 76% (683/898) 2024-08-20T21:52:48.4707182Z remote: Compressing objects: 77% (692/898) 2024-08-20T21:52:48.4714271Z remote: Compressing objects: 78% (701/898) 2024-08-20T21:52:48.4722176Z remote: Compressing objects: 79% (710/898) 2024-08-20T21:52:48.4742655Z remote: Compressing objects: 80% (719/898) 2024-08-20T21:52:48.4743270Z remote: Compressing objects: 81% (728/898) 2024-08-20T21:52:48.4755962Z remote: Compressing objects: 82% (737/898) 2024-08-20T21:52:48.4757500Z remote: Compressing objects: 83% (746/898) 2024-08-20T21:52:48.4759201Z remote: Compressing objects: 84% (755/898) 2024-08-20T21:52:48.4763511Z remote: Compressing objects: 85% (764/898) 2024-08-20T21:52:48.4771707Z remote: Compressing objects: 86% (773/898) 2024-08-20T21:52:48.4776293Z remote: Compressing objects: 87% (782/898) 2024-08-20T21:52:48.4780908Z remote: Compressing objects: 88% (791/898) 2024-08-20T21:52:48.4784547Z remote: Compressing objects: 89% (800/898) 2024-08-20T21:52:48.4790816Z remote: Compressing objects: 90% (809/898) 2024-08-20T21:52:48.4794365Z remote: Compressing objects: 91% (818/898) 2024-08-20T21:52:48.4799070Z remote: Compressing objects: 92% (827/898) 2024-08-20T21:52:48.4802324Z remote: Compressing objects: 93% (836/898) 2024-08-20T21:52:48.4804806Z remote: Compressing objects: 94% (845/898) 2024-08-20T21:52:48.4808073Z remote: Compressing objects: 95% (854/898) 2024-08-20T21:52:48.4813595Z remote: Compressing objects: 96% (863/898) 2024-08-20T21:52:48.4816493Z remote: Compressing objects: 97% (872/898) 2024-08-20T21:52:48.4818098Z remote: Compressing objects: 98% (881/898) 2024-08-20T21:52:48.4821126Z remote: Compressing objects: 99% (890/898) 2024-08-20T21:52:48.4821660Z remote: Compressing objects: 100% (898/898) 2024-08-20T21:52:48.4822230Z remote: Compressing objects: 100% (898/898), done. 2024-08-20T21:52:48.4902313Z Receiving objects: 0% (1/30997) 2024-08-20T21:52:48.4958519Z Receiving objects: 1% (310/30997) 2024-08-20T21:52:48.4999105Z Receiving objects: 2% (620/30997) 2024-08-20T21:52:48.5032715Z Receiving objects: 3% (930/30997) 2024-08-20T21:52:48.5063650Z Receiving objects: 4% (1240/30997) 2024-08-20T21:52:48.5090290Z Receiving objects: 5% (1550/30997) 2024-08-20T21:52:48.5125399Z Receiving objects: 6% (1860/30997) 2024-08-20T21:52:48.5163268Z Receiving objects: 7% (2170/30997) 2024-08-20T21:52:48.5191541Z Receiving objects: 8% (2480/30997) 2024-08-20T21:52:48.5221919Z Receiving objects: 9% (2790/30997) 2024-08-20T21:52:48.5284640Z Receiving objects: 10% (3100/30997) 2024-08-20T21:52:48.5329682Z Receiving objects: 11% (3410/30997) 2024-08-20T21:52:48.5396570Z Receiving objects: 12% (3720/30997) 2024-08-20T21:52:48.5521007Z Receiving objects: 13% (4030/30997) 2024-08-20T21:52:48.5648407Z Receiving objects: 14% (4340/30997) 2024-08-20T21:52:48.5719604Z Receiving objects: 15% (4650/30997) 2024-08-20T21:52:48.5787825Z Receiving objects: 16% (4960/30997) 2024-08-20T21:52:48.5842888Z Receiving objects: 17% (5270/30997) 2024-08-20T21:52:48.5867065Z Receiving objects: 18% (5580/30997) 2024-08-20T21:52:48.5923015Z Receiving objects: 19% (5890/30997) 2024-08-20T21:52:48.5953600Z Receiving objects: 20% (6200/30997) 2024-08-20T21:52:48.5965008Z Receiving objects: 21% (6510/30997) 2024-08-20T21:52:48.6117705Z Receiving objects: 22% (6820/30997) 2024-08-20T21:52:49.4363236Z Receiving objects: 23% (7130/30997) 2024-08-20T21:52:49.4831072Z Receiving objects: 24% (7440/30997), 43.03 MiB | 86.05 MiB/s 2024-08-20T21:52:49.9080660Z Receiving objects: 24% (7506/30997), 43.03 MiB | 86.05 MiB/s 2024-08-20T21:52:49.9683158Z Receiving objects: 25% (7750/30997), 91.15 MiB | 91.14 MiB/s 2024-08-20T21:52:50.0290105Z Receiving objects: 26% (8060/30997), 91.15 MiB | 91.14 MiB/s 2024-08-20T21:52:50.0885872Z Receiving objects: 27% (8370/30997), 131.85 MiB | 87.90 MiB/s 2024-08-20T21:52:50.1484630Z Receiving objects: 28% (8680/30997), 131.85 MiB | 87.90 MiB/s 2024-08-20T21:52:50.2088747Z Receiving objects: 29% (8990/30997), 131.85 MiB | 87.90 MiB/s 2024-08-20T21:52:50.2692587Z Receiving objects: 30% (9300/30997), 131.85 MiB | 87.90 MiB/s 2024-08-20T21:52:50.3296434Z Receiving objects: 31% (9610/30997), 131.85 MiB | 87.90 MiB/s 2024-08-20T21:52:50.4835324Z Receiving objects: 32% (9920/30997), 131.85 MiB | 87.90 MiB/s 2024-08-20T21:52:50.5313383Z Receiving objects: 32% (10172/30997), 177.90 MiB | 88.94 MiB/s 2024-08-20T21:52:50.5995028Z Receiving objects: 33% (10230/30997), 177.90 MiB | 88.94 MiB/s 2024-08-20T21:52:50.6110815Z Receiving objects: 34% (10539/30997), 177.90 MiB | 88.94 MiB/s 2024-08-20T21:52:50.8412186Z Receiving objects: 35% (10849/30997), 177.90 MiB | 88.94 MiB/s 2024-08-20T21:52:51.0485034Z Receiving objects: 36% (11159/30997), 177.90 MiB | 88.94 MiB/s 2024-08-20T21:52:51.0504233Z Receiving objects: 37% (11469/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0523601Z Receiving objects: 38% (11779/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0541667Z Receiving objects: 39% (12089/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0556002Z Receiving objects: 40% (12399/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0561143Z Receiving objects: 41% (12709/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0573222Z Receiving objects: 42% (13019/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0590610Z Receiving objects: 43% (13329/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0603277Z Receiving objects: 44% (13639/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0609368Z Receiving objects: 45% (13949/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0613511Z Receiving objects: 46% (14259/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0630478Z Receiving objects: 47% (14569/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0646871Z Receiving objects: 48% (14879/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0662662Z Receiving objects: 49% (15189/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0845090Z Receiving objects: 50% (15499/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0849898Z Receiving objects: 51% (15809/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0855861Z Receiving objects: 52% (16119/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0861578Z Receiving objects: 53% (16429/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0866630Z Receiving objects: 54% (16739/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0883788Z Receiving objects: 55% (17049/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0891907Z Receiving objects: 56% (17359/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0897475Z Receiving objects: 57% (17669/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0914263Z Receiving objects: 58% (17979/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0921592Z Receiving objects: 59% (18289/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.0930731Z Receiving objects: 60% (18599/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1045943Z Receiving objects: 61% (18909/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1050499Z Receiving objects: 62% (19219/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1055627Z Receiving objects: 63% (19529/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1062742Z Receiving objects: 64% (19839/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1069589Z Receiving objects: 65% (20149/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1077344Z Receiving objects: 66% (20459/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1299854Z Receiving objects: 67% (20768/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1669368Z Receiving objects: 68% (21078/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1672136Z Receiving objects: 69% (21388/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1685192Z Receiving objects: 70% (21698/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1694980Z Receiving objects: 71% (22008/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1712324Z Receiving objects: 72% (22318/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1716615Z Receiving objects: 73% (22628/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1729388Z Receiving objects: 74% (22938/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1740135Z Receiving objects: 75% (23248/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1746493Z Receiving objects: 76% (23558/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1753551Z Receiving objects: 77% (23868/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1960559Z Receiving objects: 78% (24178/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1992379Z Receiving objects: 79% (24488/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.1996710Z Receiving objects: 80% (24798/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2007408Z Receiving objects: 81% (25108/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2017150Z Receiving objects: 82% (25418/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2066065Z Receiving objects: 83% (25728/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2182341Z Receiving objects: 84% (26038/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2183936Z Receiving objects: 85% (26348/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2190921Z Receiving objects: 86% (26658/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2195466Z Receiving objects: 87% (26968/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2202324Z Receiving objects: 88% (27278/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2207156Z Receiving objects: 89% (27588/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2317877Z Receiving objects: 90% (27898/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2326750Z Receiving objects: 91% (28208/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2337195Z Receiving objects: 92% (28518/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2357508Z Receiving objects: 93% (28828/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2387405Z Receiving objects: 94% (29138/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2395205Z Receiving objects: 95% (29448/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2402201Z Receiving objects: 96% (29758/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2406570Z Receiving objects: 97% (30068/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2474745Z Receiving objects: 98% (30378/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2498985Z Receiving objects: 99% (30688/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2500070Z remote: Total 30997 (delta 1010), reused 1469 (delta 820), pack-reused 29260 (from 1) 2024-08-20T21:52:51.2517100Z Receiving objects: 100% (30997/30997), 215.29 MiB | 86.15 MiB/s 2024-08-20T21:52:51.2517834Z Receiving objects: 100% (30997/30997), 232.50 MiB | 83.98 MiB/s, done. 2024-08-20T21:52:51.2544678Z Resolving deltas: 0% (0/16723) 2024-08-20T21:52:51.2548702Z Resolving deltas: 1% (168/16723) 2024-08-20T21:52:51.2551212Z Resolving deltas: 2% (335/16723) 2024-08-20T21:52:51.2556081Z Resolving deltas: 3% (502/16723) 2024-08-20T21:52:51.2560289Z Resolving deltas: 4% (669/16723) 2024-08-20T21:52:51.2562597Z Resolving deltas: 5% (837/16723) 2024-08-20T21:52:51.2565299Z Resolving deltas: 6% (1005/16723) 2024-08-20T21:52:51.2569318Z Resolving deltas: 7% (1171/16723) 2024-08-20T21:52:51.2572700Z Resolving deltas: 8% (1338/16723) 2024-08-20T21:52:51.2576008Z Resolving deltas: 9% (1506/16723) 2024-08-20T21:52:51.2579143Z Resolving deltas: 10% (1673/16723) 2024-08-20T21:52:51.2582904Z Resolving deltas: 11% (1840/16723) 2024-08-20T21:52:51.2586202Z Resolving deltas: 12% (2008/16723) 2024-08-20T21:52:51.2590093Z Resolving deltas: 13% (2174/16723) 2024-08-20T21:52:51.2594318Z Resolving deltas: 14% (2342/16723) 2024-08-20T21:52:51.2597839Z Resolving deltas: 15% (2509/16723) 2024-08-20T21:52:51.2601643Z Resolving deltas: 16% (2676/16723) 2024-08-20T21:52:51.2605429Z Resolving deltas: 17% (2843/16723) 2024-08-20T21:52:51.2608847Z Resolving deltas: 18% (3011/16723) 2024-08-20T21:52:51.2612669Z Resolving deltas: 19% (3178/16723) 2024-08-20T21:52:51.2618403Z Resolving deltas: 20% (3345/16723) 2024-08-20T21:52:51.2624269Z Resolving deltas: 21% (3512/16723) 2024-08-20T21:52:51.2631831Z Resolving deltas: 22% (3680/16723) 2024-08-20T21:52:51.2638991Z Resolving deltas: 23% (3847/16723) 2024-08-20T21:52:51.2643236Z Resolving deltas: 24% (4014/16723) 2024-08-20T21:52:51.2649898Z Resolving deltas: 25% (4181/16723) 2024-08-20T21:52:51.2654790Z Resolving deltas: 26% (4348/16723) 2024-08-20T21:52:51.2659365Z Resolving deltas: 27% (4516/16723) 2024-08-20T21:52:51.2669675Z Resolving deltas: 28% (4683/16723) 2024-08-20T21:52:51.2673792Z Resolving deltas: 29% (4850/16723) 2024-08-20T21:52:51.2677046Z Resolving deltas: 30% (5019/16723) 2024-08-20T21:52:51.2679715Z Resolving deltas: 31% (5185/16723) 2024-08-20T21:52:51.2682376Z Resolving deltas: 32% (5352/16723) 2024-08-20T21:52:51.2684989Z Resolving deltas: 33% (5519/16723) 2024-08-20T21:52:51.2687820Z Resolving deltas: 34% (5686/16723) 2024-08-20T21:52:51.2690554Z Resolving deltas: 35% (5854/16723) 2024-08-20T21:52:51.2694295Z Resolving deltas: 36% (6021/16723) 2024-08-20T21:52:51.2697427Z Resolving deltas: 37% (6188/16723) 2024-08-20T21:52:51.2701640Z Resolving deltas: 38% (6356/16723) 2024-08-20T21:52:51.2709547Z Resolving deltas: 39% (6522/16723) 2024-08-20T21:52:51.2715044Z Resolving deltas: 40% (6690/16723) 2024-08-20T21:52:51.2721756Z Resolving deltas: 41% (6857/16723) 2024-08-20T21:52:51.2727811Z Resolving deltas: 42% (7024/16723) 2024-08-20T21:52:51.2730913Z Resolving deltas: 43% (7191/16723) 2024-08-20T21:52:51.2733983Z Resolving deltas: 44% (7359/16723) 2024-08-20T21:52:51.2736869Z Resolving deltas: 45% (7526/16723) 2024-08-20T21:52:51.2740630Z Resolving deltas: 46% (7693/16723) 2024-08-20T21:52:51.2758695Z Resolving deltas: 47% (7860/16723) 2024-08-20T21:52:51.2762115Z Resolving deltas: 48% (8028/16723) 2024-08-20T21:52:51.2767199Z Resolving deltas: 49% (8195/16723) 2024-08-20T21:52:51.2774716Z Resolving deltas: 50% (8362/16723) 2024-08-20T21:52:51.2782227Z Resolving deltas: 51% (8529/16723) 2024-08-20T21:52:51.2787571Z Resolving deltas: 52% (8696/16723) 2024-08-20T21:52:51.2792445Z Resolving deltas: 53% (8864/16723) 2024-08-20T21:52:51.2796218Z Resolving deltas: 54% (9031/16723) 2024-08-20T21:52:51.2799737Z Resolving deltas: 55% (9198/16723) 2024-08-20T21:52:51.2803449Z Resolving deltas: 56% (9365/16723) 2024-08-20T21:52:51.2807098Z Resolving deltas: 57% (9533/16723) 2024-08-20T21:52:51.2811470Z Resolving deltas: 58% (9700/16723) 2024-08-20T21:52:51.2817960Z Resolving deltas: 59% (9868/16723) 2024-08-20T21:52:51.2822708Z Resolving deltas: 60% (10034/16723) 2024-08-20T21:52:51.2828735Z Resolving deltas: 61% (10202/16723) 2024-08-20T21:52:51.2834972Z Resolving deltas: 62% (10369/16723) 2024-08-20T21:52:51.2840826Z Resolving deltas: 63% (10536/16723) 2024-08-20T21:52:51.2849004Z Resolving deltas: 64% (10703/16723) 2024-08-20T21:52:51.2856473Z Resolving deltas: 65% (10870/16723) 2024-08-20T21:52:51.2862172Z Resolving deltas: 66% (11038/16723) 2024-08-20T21:52:51.2867427Z Resolving deltas: 67% (11205/16723) 2024-08-20T21:52:51.2869930Z Resolving deltas: 68% (11372/16723) 2024-08-20T21:52:51.2875435Z Resolving deltas: 69% (11539/16723) 2024-08-20T21:52:51.2880571Z Resolving deltas: 70% (11707/16723) 2024-08-20T21:52:51.2885047Z Resolving deltas: 71% (11874/16723) 2024-08-20T21:52:51.2888571Z Resolving deltas: 72% (12041/16723) 2024-08-20T21:52:51.2893230Z Resolving deltas: 73% (12208/16723) 2024-08-20T21:52:51.2900933Z Resolving deltas: 74% (12376/16723) 2024-08-20T21:52:51.2906964Z Resolving deltas: 75% (12543/16723) 2024-08-20T21:52:51.2911642Z Resolving deltas: 76% (12710/16723) 2024-08-20T21:52:51.2914982Z Resolving deltas: 77% (12877/16723) 2024-08-20T21:52:51.2922166Z Resolving deltas: 78% (13044/16723) 2024-08-20T21:52:51.2927992Z Resolving deltas: 79% (13212/16723) 2024-08-20T21:52:51.2934210Z Resolving deltas: 80% (13380/16723) 2024-08-20T21:52:51.2939523Z Resolving deltas: 81% (13546/16723) 2024-08-20T21:52:51.2945116Z Resolving deltas: 82% (13713/16723) 2024-08-20T21:52:51.2951766Z Resolving deltas: 83% (13881/16723) 2024-08-20T21:52:51.2957995Z Resolving deltas: 84% (14048/16723) 2024-08-20T21:52:51.2961465Z Resolving deltas: 85% (14215/16723) 2024-08-20T21:52:51.2965305Z Resolving deltas: 86% (14382/16723) 2024-08-20T21:52:51.2968402Z Resolving deltas: 87% (14550/16723) 2024-08-20T21:52:51.2972229Z Resolving deltas: 88% (14717/16723) 2024-08-20T21:52:51.2977079Z Resolving deltas: 89% (14884/16723) 2024-08-20T21:52:51.2983109Z Resolving deltas: 90% (15051/16723) 2024-08-20T21:52:51.2987429Z Resolving deltas: 91% (15218/16723) 2024-08-20T21:52:51.2990768Z Resolving deltas: 92% (15387/16723) 2024-08-20T21:52:51.2994421Z Resolving deltas: 93% (15553/16723) 2024-08-20T21:52:51.3000676Z Resolving deltas: 94% (15720/16723) 2024-08-20T21:52:51.3007202Z Resolving deltas: 95% (15887/16723) 2024-08-20T21:52:51.3012012Z Resolving deltas: 96% (16055/16723) 2024-08-20T21:52:51.3017678Z Resolving deltas: 97% (16222/16723) 2024-08-20T21:52:51.3022169Z Resolving deltas: 98% (16389/16723) 2024-08-20T21:52:51.3143989Z Resolving deltas: 99% (16556/16723) 2024-08-20T21:52:51.3144454Z Resolving deltas: 100% (16723/16723) 2024-08-20T21:52:51.3144874Z Resolving deltas: 100% (16723/16723), done. 2024-08-20T21:52:52.2745594Z + pushd torchbench 2024-08-20T21:52:52.2746110Z ~/workspace/torchbench ~/workspace 2024-08-20T21:52:52.2746621Z + git checkout 23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T21:52:52.3348607Z Note: switching to '23512dbebd44a11eb84afbf53c3c071dd105297e'. 2024-08-20T21:52:52.3349050Z 2024-08-20T21:52:52.3349521Z You are in 'detached HEAD' state. You can look around, make experimental 2024-08-20T21:52:52.3350364Z changes and commit them, and you can discard any commits you make in this 2024-08-20T21:52:52.3351146Z state without impacting any branches by switching back to a branch. 2024-08-20T21:52:52.3351587Z 2024-08-20T21:52:52.3351916Z If you want to create a new branch to retain commits you create, you may 2024-08-20T21:52:52.3352724Z do so (now or later) by using -c with the switch command. Example: 2024-08-20T21:52:52.3353146Z 2024-08-20T21:52:52.3353322Z git switch -c 2024-08-20T21:52:52.3353599Z 2024-08-20T21:52:52.3353723Z Or undo this operation with: 2024-08-20T21:52:52.3353971Z 2024-08-20T21:52:52.3354073Z git switch - 2024-08-20T21:52:52.3354240Z 2024-08-20T21:52:52.3354587Z Turn off this advice by setting config variable advice.detachedHead to false 2024-08-20T21:52:52.3355076Z 2024-08-20T21:52:52.3355238Z HEAD is now at 23512dbe Pin numpy version 2024-08-20T21:52:52.3355665Z + '[' '' ']' 2024-08-20T21:52:52.3356014Z + python install.py --continue_on_fail 2024-08-20T21:52:54.3448215Z checking packages torch, torchvision, torchaudio are installed...OK 2024-08-20T21:53:09.2400564Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch...OK 2024-08-20T21:53:10.8768058Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Background_Matting...OK 2024-08-20T21:53:12.7820877Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/LearningToPaint...OK 2024-08-20T21:53:14.4364827Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Super_SloMo...OK 2024-08-20T21:53:14.4666677Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/alexnet...OK 2024-08-20T21:53:20.9035943Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_edgecnn...OK 2024-08-20T21:53:23.5345789Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gcn...OK 2024-08-20T21:53:26.0321007Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gin...OK 2024-08-20T21:53:28.5184371Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_sage...OK 2024-08-20T21:53:28.5186377Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/cm3leon_generate...SKIP - No install.py is found 2024-08-20T21:53:29.6086194Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dcgan...OK 2024-08-20T21:53:31.8400303Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/demucs...OK 2024-08-20T21:53:31.8704411Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/densenet121...OK 2024-08-20T21:54:15.1161265Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_c4...OK 2024-08-20T21:54:23.0128259Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_dc5...OK 2024-08-20T21:54:29.5598578Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_fpn...OK 2024-08-20T21:54:35.6198771Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_c4...OK 2024-08-20T21:54:43.1339679Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_dc5...OK 2024-08-20T21:54:49.1865751Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_fpn...OK 2024-08-20T21:54:55.6277383Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fcos_r_50_fpn...OK 2024-08-20T21:55:02.0540633Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn...OK 2024-08-20T21:55:08.2852229Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_c4...OK 2024-08-20T21:55:15.5631300Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_fpn...OK 2024-08-20T21:55:21.5849900Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_c4...OK 2024-08-20T21:55:27.7247841Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_fpn...OK 2024-08-20T21:55:29.0564521Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dlrm...OK 2024-08-20T21:56:02.3445102Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_det_predictor...OK 2024-08-20T21:56:06.9233788Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_reco_predictor...OK 2024-08-20T21:56:11.9629619Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/drq...OK 2024-08-20T22:01:55.9234686Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/fastNLP_Bert...OK 2024-08-20T22:01:57.1179558Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_dp_cifar10...OK 2024-08-20T22:01:58.5832316Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_maml_omniglot...OK 2024-08-20T22:02:04.8640975Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Albert...OK 2024-08-20T22:02:11.6449673Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bart...OK 2024-08-20T22:02:16.8552626Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert...OK 2024-08-20T22:02:24.3083316Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert_large...OK 2024-08-20T22:02:29.9512550Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_BigBird...OK 2024-08-20T22:02:34.7475317Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_DistilBert...OK 2024-08-20T22:02:40.6869920Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2...OK 2024-08-20T22:02:56.6562803Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2_large...OK 2024-08-20T22:03:02.5872836Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Longformer...OK 2024-08-20T22:03:06.2511668Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Reformer...OK 2024-08-20T22:03:10.7854963Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5...OK 2024-08-20T22:03:17.2982213Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_base...OK 2024-08-20T22:03:17.2983902Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_generate...SKIP - No install.py is found 2024-08-20T22:03:30.2010332Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_large...OK 2024-08-20T22:03:33.8810510Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Whisper...OK 2024-08-20T22:03:33.8813390Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_clip...SKIP - No install.py is found 2024-08-20T22:03:41.3619221Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_distil_whisper...OK 2024-08-20T22:03:42.6103045Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/lennard_jones...OK 2024-08-20T22:03:43.8624722Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama...OK 2024-08-20T22:04:28.1769384Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama_v2_7b_16h...OK 2024-08-20T22:05:56.3857323Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llava...OK 2024-08-20T22:05:56.4168671Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml...OK 2024-08-20T22:05:57.8997292Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml_omniglot...OK 2024-08-20T22:05:57.9306474Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mnasnet1_0...OK 2024-08-20T22:05:57.9595916Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2...OK 2024-08-20T22:05:57.9865159Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2_quantized_qat...OK 2024-08-20T22:05:58.0132305Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v3_large...OK 2024-08-20T22:05:58.0404148Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco...OK 2024-08-20T22:06:18.8266843Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moondream...OK 2024-08-20T22:06:18.8268423Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nanogpt...SKIP - No install.py is found 2024-08-20T22:06:20.0942272Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nvidia_deeprecommender...OK 2024-08-20T22:06:22.4835143Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/opacus_cifar10...OK 2024-08-20T22:06:22.5142851Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_densenet...OK 2024-08-20T22:06:22.5427940Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_resnet...OK 2024-08-20T22:06:22.5707102Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_equation_of_state...OK 2024-08-20T22:06:22.5969069Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_isoneutral_mixing...OK 2024-08-20T22:06:22.6243539Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_turbulent_kinetic_energy...OK 2024-08-20T22:06:28.2494843Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_CycleGAN_and_pix2pix...OK 2024-08-20T22:06:29.7016170Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_stargan...OK 2024-08-20T22:06:33.0147498Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_unet...OK 2024-08-20T22:06:33.0462957Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet152...OK 2024-08-20T22:06:33.0748831Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet18...OK 2024-08-20T22:06:33.1026356Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50...OK 2024-08-20T22:06:33.1293021Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50_quantized_qat...OK 2024-08-20T22:06:33.1564286Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnext50_32x4d...OK 2024-08-20T22:06:43.8285115Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam...OK 2024-08-20T22:06:55.5010061Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam_fast...OK 2024-08-20T22:06:55.5316585Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/shufflenet_v2_x1_0...OK 2024-08-20T22:06:55.5318256Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt...SKIP - No install.py is found 2024-08-20T22:06:55.5319901Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt_tp_manual...SKIP - No install.py is found 2024-08-20T22:06:58.0297458Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/soft_actor_critic...OK 2024-08-20T22:07:00.0554691Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/speech_transformer...OK 2024-08-20T22:07:00.0867562Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/squeezenet1_1...OK 2024-08-20T22:07:25.6765063Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_text_encoder...OK 2024-08-20T22:07:39.5123173Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_unet...OK 2024-08-20T22:07:42.2767475Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tacotron2...OK 2024-08-20T22:07:47.8994745Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientdet...OK 2024-08-20T22:07:47.9311791Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientnet...OK 2024-08-20T22:07:47.9596667Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_nfnet...OK 2024-08-20T22:07:47.9871900Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_regnet...OK 2024-08-20T22:07:48.0142162Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_resnest...OK 2024-08-20T22:07:48.0408220Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer...OK 2024-08-20T22:07:48.0674253Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer_large...OK 2024-08-20T22:07:48.0948666Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vovnet...OK 2024-08-20T22:07:57.8428551Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/torch_multimodal_clip...OK 2024-08-20T22:08:14.7342222Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tts_angular...OK 2024-08-20T22:08:14.7659951Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vgg16...OK 2024-08-20T22:08:16.8812885Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vision_maskrcnn...OK 2024-08-20T22:08:18.8202750Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/yolov3...OK 2024-08-20T22:08:19.1948264Z + echo 'Print all dependencies after TorchBench is installed' 2024-08-20T22:08:19.1948989Z Print all dependencies after TorchBench is installed 2024-08-20T22:08:19.6558820Z + python -mpip freeze 2024-08-20T22:08:19.6559364Z absl-py==2.1.0 2024-08-20T22:08:19.6559922Z accelerate==0.33.0 2024-08-20T22:08:19.6560309Z aiohappyeyeballs==2.4.0 2024-08-20T22:08:19.6560907Z aiohttp==3.10.5 2024-08-20T22:08:19.6561189Z aiosignal==1.3.1 2024-08-20T22:08:19.6561461Z alabaster==0.7.13 2024-08-20T22:08:19.6561913Z annotated-types==0.7.0 2024-08-20T22:08:19.6562301Z antlr4-python3-runtime==4.9.3 2024-08-20T22:08:19.6562660Z anyascii==0.3.2 2024-08-20T22:08:19.6562933Z astroid==3.2.4 2024-08-20T22:08:19.6563224Z asttokens==2.4.1 2024-08-20T22:08:19.6563677Z astunparse==1.6.3 2024-08-20T22:08:19.6564344Z async-timeout==4.0.3 2024-08-20T22:08:19.6564841Z attrs==23.1.0 2024-08-20T22:08:19.6565322Z audioread==3.0.1 2024-08-20T22:08:19.6565672Z babel==2.16.0 2024-08-20T22:08:19.6565927Z backcall==0.2.0 2024-08-20T22:08:19.6566231Z beautifulsoup4==4.12.3 2024-08-20T22:08:19.6567385Z -e git+https://github.com/pytorch/benchmark@23512dbebd44a11eb84afbf53c3c071dd105297e#egg=bert_pytorch&subdirectory=torchbenchmark/models/BERT_pytorch 2024-08-20T22:08:19.6568620Z bibtexparser==2.0.0b7 2024-08-20T22:08:19.6568983Z black==24.8.0 2024-08-20T22:08:19.6569251Z blinker==1.8.2 2024-08-20T22:08:19.6569514Z blis==0.7.11 2024-08-20T22:08:19.6569778Z blobfile==2.1.1 2024-08-20T22:08:19.6570046Z bokeh==3.1.1 2024-08-20T22:08:19.6570305Z boto3==1.19.12 2024-08-20T22:08:19.6570558Z botocore==1.22.12 2024-08-20T22:08:19.6570838Z breathe==4.34.0 2024-08-20T22:08:19.6571106Z bs4==0.0.1 2024-08-20T22:08:19.6571352Z cachetools==5.5.0 2024-08-20T22:08:19.6571652Z cardboardlint==1.3.1 2024-08-20T22:08:19.6571955Z catalogue==2.0.10 2024-08-20T22:08:19.6572222Z certifi==2024.7.4 2024-08-20T22:08:19.6572491Z cffi==1.17.0 2024-08-20T22:08:19.6572816Z charset-normalizer==3.3.2 2024-08-20T22:08:19.6573128Z click==8.1.7 2024-08-20T22:08:19.6573390Z clldutils==3.22.2 2024-08-20T22:08:19.6573677Z cloudpathlib==0.18.1 2024-08-20T22:08:19.6573981Z cloudpickle==3.0.0 2024-08-20T22:08:19.6574255Z colorama==0.4.6 2024-08-20T22:08:19.6574536Z colorlog==6.8.2 2024-08-20T22:08:19.6574808Z comm==0.2.2 2024-08-20T22:08:19.6575053Z confection==0.1.5 2024-08-20T22:08:19.6575336Z contourpy==1.1.1 2024-08-20T22:08:19.6575625Z coremltools==5.0b5 2024-08-20T22:08:19.6575918Z cryptography==43.0.0 2024-08-20T22:08:19.6576200Z csvw==3.3.0 2024-08-20T22:08:19.6576450Z cycler==0.12.1 2024-08-20T22:08:19.6576713Z cymem==2.0.8 2024-08-20T22:08:19.6576987Z DALL-E==0.1 2024-08-20T22:08:19.6577242Z datasets==2.21.0 2024-08-20T22:08:19.6577517Z debugpy==1.8.5 2024-08-20T22:08:19.6577786Z decorator==5.1.1 2024-08-20T22:08:19.6578053Z defusedxml==0.7.1 2024-08-20T22:08:19.6578336Z Deprecated==1.2.14 2024-08-20T22:08:19.6579055Z detectron2 @ git+https://github.com/facebookresearch/detectron2.git@1a4df4d954089534daebbd7a0a45f08ece87056e 2024-08-20T22:08:19.6579864Z diffusers==0.20.2 2024-08-20T22:08:19.6580131Z dill==0.3.8 2024-08-20T22:08:19.6580385Z diskcache==5.6.3 2024-08-20T22:08:19.6580664Z distro==1.9.0 2024-08-20T22:08:19.6580915Z dlinfo==1.2.1 2024-08-20T22:08:19.6581190Z dnspython==2.6.1 2024-08-20T22:08:19.6581523Z docker-pycreds==0.4.0 2024-08-20T22:08:19.6581834Z docutils==0.16 2024-08-20T22:08:19.6582096Z dominate==2.9.1 2024-08-20T22:08:19.6582919Z effdet @ git+https://github.com/rwightman/efficientdet-pytorch.git@d43c9e34cd62d22b4205831bb735f6dd83b8e881 2024-08-20T22:08:19.6583721Z einops==0.8.0 2024-08-20T22:08:19.6583997Z exceptiongroup==1.2.2 2024-08-20T22:08:19.6584307Z execnet==2.1.1 2024-08-20T22:08:19.6584587Z executing==2.0.1 2024-08-20T22:08:19.6584866Z exhale==0.2.3 2024-08-20T22:08:19.6585126Z expecttest==0.1.6 2024-08-20T22:08:19.6585423Z fastjsonschema==2.20.0 2024-08-20T22:08:19.6585741Z FastNLP==0.6.0 2024-08-20T22:08:19.6586045Z ffmpeg-python==0.2.0 2024-08-20T22:08:19.6586353Z filelock==3.13.1 2024-08-20T22:08:19.6586632Z Flask==3.0.3 2024-08-20T22:08:19.6586903Z flatbuffers==2.0 2024-08-20T22:08:19.6587176Z fonttools==4.53.1 2024-08-20T22:08:19.6587463Z frozenlist==1.4.1 2024-08-20T22:08:19.6587747Z fsspec==2024.6.1 2024-08-20T22:08:19.6588016Z ftfy==6.2.3 2024-08-20T22:08:19.6588623Z functorch @ git+https://github.com/pytorch/functorch.git@b71aa0b4387b86c278132209b99538be48ef4c74 2024-08-20T22:08:19.6589477Z future==1.0.0 2024-08-20T22:08:19.6589763Z fvcore==0.1.5.post20221221 2024-08-20T22:08:19.6590076Z gdown==5.2.0 2024-08-20T22:08:19.6590344Z geojson==2.5.0 2024-08-20T22:08:19.6590616Z ghstack==0.8.0 2024-08-20T22:08:19.6590887Z gitdb==4.0.11 2024-08-20T22:08:19.6591146Z GitPython==3.1.43 2024-08-20T22:08:19.6591473Z google-auth==2.34.0 2024-08-20T22:08:19.6591824Z google-auth-oauthlib==1.0.0 2024-08-20T22:08:19.6592155Z greenlet==3.0.3 2024-08-20T22:08:19.6592425Z grpcio==1.65.5 2024-08-20T22:08:19.6592688Z gym==0.26.2 2024-08-20T22:08:19.6592974Z gym-notices==0.0.8 2024-08-20T22:08:19.6593241Z h5py==3.11.0 2024-08-20T22:08:19.6593497Z higher==0.2.1 2024-08-20T22:08:19.6593808Z huggingface-hub==0.24.6 2024-08-20T22:08:19.6594140Z hydra-core==1.3.2 2024-08-20T22:08:19.6594424Z hypothesis==5.35.1 2024-08-20T22:08:19.6594709Z idna==3.7 2024-08-20T22:08:19.6595030Z imageio==2.35.1 2024-08-20T22:08:19.6595291Z imagesize==1.4.1 2024-08-20T22:08:19.6595585Z importlib_metadata==8.3.0 2024-08-20T22:08:19.6595927Z importlib_resources==6.4.3 2024-08-20T22:08:19.6596244Z inflect==7.3.1 2024-08-20T22:08:19.6596518Z iniconfig==2.0.0 2024-08-20T22:08:19.6596796Z iopath==0.1.9 2024-08-20T22:08:19.6597067Z ipykernel==6.29.5 2024-08-20T22:08:19.6597337Z ipython==8.12.0 2024-08-20T22:08:19.6597610Z isodate==0.6.1 2024-08-20T22:08:19.6597875Z isort==5.13.2 2024-08-20T22:08:19.6598134Z itsdangerous==2.2.0 2024-08-20T22:08:19.6598423Z jedi==0.19.1 2024-08-20T22:08:19.6598681Z Jinja2==3.1.4 2024-08-20T22:08:19.6598945Z jmespath==0.10.0 2024-08-20T22:08:19.6599209Z joblib==1.4.2 2024-08-20T22:08:19.6599475Z jsonpatch==1.33 2024-08-20T22:08:19.6599754Z jsonpointer==3.0.0 2024-08-20T22:08:19.6600046Z jsonschema==4.23.0 2024-08-20T22:08:19.6600413Z jsonschema-specifications==2023.12.1 2024-08-20T22:08:19.6600817Z junitparser==2.1.1 2024-08-20T22:08:19.6601147Z jupyter-cache==0.6.1 2024-08-20T22:08:19.6601447Z jupyter_client==8.6.2 2024-08-20T22:08:19.6601854Z jupyter_core==5.7.2 2024-08-20T22:08:19.6602183Z kaldi-io==0.9.8 2024-08-20T22:08:19.6602469Z kiwisolver==1.4.5 2024-08-20T22:08:19.6602737Z kornia==0.7.3 2024-08-20T22:08:19.6603006Z kornia_rs==0.1.5 2024-08-20T22:08:19.6603292Z lameenc==1.7.0 2024-08-20T22:08:19.6603554Z langcodes==3.4.0 2024-08-20T22:08:19.6603836Z langdetect==1.0.9 2024-08-20T22:08:19.6604158Z language-tags==1.2.0 2024-08-20T22:08:19.6604459Z language_data==1.2.0 2024-08-20T22:08:19.6604736Z lark==0.12.0 2024-08-20T22:08:19.6604994Z libcst==1.1.0 2024-08-20T22:08:19.6605259Z librosa==0.9.2 2024-08-20T22:08:19.6605520Z lintrunner==0.12.5 2024-08-20T22:08:19.6605807Z llvmlite==0.32.1 2024-08-20T22:08:19.6606080Z lxml==4.9.4 2024-08-20T22:08:19.6606366Z marisa-trie==1.2.0 2024-08-20T22:08:19.6606638Z Markdown==3.7 2024-08-20T22:08:19.6606946Z markdown-it-py==2.2.0 2024-08-20T22:08:19.6607253Z MarkupSafe==2.1.5 2024-08-20T22:08:19.6607527Z matplotlib==3.5.3 2024-08-20T22:08:19.6607865Z matplotlib-inline==0.1.7 2024-08-20T22:08:19.6608187Z mccabe==0.7.0 2024-08-20T22:08:19.6608501Z mdit-py-plugins==0.3.5 2024-08-20T22:08:19.6608800Z mdurl==0.1.2 2024-08-20T22:08:19.6609090Z ml-dtypes==0.2.0 2024-08-20T22:08:19.6609373Z MonkeyType==23.3.0 2024-08-20T22:08:19.6609687Z more-itertools==10.4.0 2024-08-20T22:08:19.6609995Z mpmath==1.3.0 2024-08-20T22:08:19.6610261Z multidict==6.0.5 2024-08-20T22:08:19.6610553Z multiprocess==0.70.16 2024-08-20T22:08:19.6610850Z murmurhash==1.0.10 2024-08-20T22:08:19.6611135Z musdb==0.4.2 2024-08-20T22:08:19.6611401Z museval==0.4.1 2024-08-20T22:08:19.6611657Z mypy==1.10.0 2024-08-20T22:08:19.6611970Z mypy-extensions==1.0.0 2024-08-20T22:08:19.6612312Z myst-nb==0.17.2 2024-08-20T22:08:19.6612622Z myst-parser==0.18.1 2024-08-20T22:08:19.6612902Z nbclient==0.7.4 2024-08-20T22:08:19.6613177Z nbformat==5.10.4 2024-08-20T22:08:19.6613496Z nest-asyncio==1.6.0 2024-08-20T22:08:19.6613796Z networkx==2.8.8 2024-08-20T22:08:19.6614058Z ninja==1.10.2 2024-08-20T22:08:19.6614327Z nose==1.3.7 2024-08-20T22:08:19.6614582Z numba==0.49.0 2024-08-20T22:08:19.6614832Z numpy==1.21.2 2024-08-20T22:08:19.6615247Z nvidia-cublas-cu12==12.1.3.1 2024-08-20T22:08:19.6615668Z nvidia-cuda-cupti-cu12==12.1.105 2024-08-20T22:08:19.6616104Z nvidia-cuda-nvrtc-cu12==12.1.105 2024-08-20T22:08:19.6616532Z nvidia-cuda-runtime-cu12==12.1.105 2024-08-20T22:08:19.6617033Z nvidia-cudnn-cu12==9.1.0.70 2024-08-20T22:08:19.6617413Z nvidia-cufft-cu12==11.0.2.54 2024-08-20T22:08:19.6617812Z nvidia-curand-cu12==10.3.2.106 2024-08-20T22:08:19.6618230Z nvidia-cusolver-cu12==11.4.5.107 2024-08-20T22:08:19.6618648Z nvidia-cusparse-cu12==12.1.0.106 2024-08-20T22:08:19.6619057Z nvidia-nccl-cu12==2.20.5 2024-08-20T22:08:19.6619438Z nvidia-nvjitlink-cu12==12.6.20 2024-08-20T22:08:19.6619842Z nvidia-nvtx-cu12==12.1.105 2024-08-20T22:08:19.6620159Z oauthlib==3.2.2 2024-08-20T22:08:19.6620439Z omegaconf==2.3.0 2024-08-20T22:08:19.6620717Z onnx==1.16.1 2024-08-20T22:08:19.6620985Z onnxscript==0.1.0.dev20240817 2024-08-20T22:08:19.6621389Z opacus==1.5.2 2024-08-20T22:08:19.6621747Z opencv-python==4.8.0.74 2024-08-20T22:08:19.6622180Z opt-einsum==3.3.0 2024-08-20T22:08:19.6622455Z optree==0.12.1 2024-08-20T22:08:19.6622753Z packaging==24.1 2024-08-20T22:08:19.6623028Z pandas==2.0.3 2024-08-20T22:08:19.6623292Z parso==0.8.4 2024-08-20T22:08:19.6623539Z patch==1.16 2024-08-20T22:08:19.6623798Z pathspec==0.12.1 2024-08-20T22:08:19.6624079Z pexpect==4.9.0 2024-08-20T22:08:19.6624340Z phonemizer==3.3.0 2024-08-20T22:08:19.6624629Z pickleshare==0.7.5 2024-08-20T22:08:19.6624913Z pillow==10.3.0 2024-08-20T22:08:19.6625199Z pkgutil_resolve_name==1.3.10 2024-08-20T22:08:19.6625533Z platformdirs==4.2.2 2024-08-20T22:08:19.6625820Z pluggy==1.5.0 2024-08-20T22:08:19.6626086Z pooch==1.8.2 2024-08-20T22:08:19.6626341Z portalocker==2.10.1 2024-08-20T22:08:19.6626629Z preshed==3.0.9 2024-08-20T22:08:19.6626905Z prettytable==3.11.0 2024-08-20T22:08:19.6627203Z prompt_toolkit==3.0.47 2024-08-20T22:08:19.6627504Z protobuf==3.20.2 2024-08-20T22:08:19.6627781Z psutil==6.0.0 2024-08-20T22:08:19.6628050Z ptyprocess==0.7.0 2024-08-20T22:08:19.6628324Z pure_eval==0.2.3 2024-08-20T22:08:19.6628639Z py-cpuinfo==9.0.0 2024-08-20T22:08:19.6628917Z pyaml==24.7.0 2024-08-20T22:08:19.6629182Z pyarrow==17.0.0 2024-08-20T22:08:19.6629440Z pyasn1==0.6.0 2024-08-20T22:08:19.6629709Z pyasn1_modules==0.4.0 2024-08-20T22:08:19.6630020Z pyclipper==1.3.0.post5 2024-08-20T22:08:19.6630324Z pycocotools==2.0.7 2024-08-20T22:08:19.6630616Z pycparser==2.22 2024-08-20T22:08:19.6630907Z pycryptodomex==3.20.0 2024-08-20T22:08:19.6631215Z pydantic==2.8.2 2024-08-20T22:08:19.6631486Z pydantic_core==2.20.1 2024-08-20T22:08:19.6631783Z pydot==3.0.1 2024-08-20T22:08:19.6632045Z pygame==2.6.0 2024-08-20T22:08:19.6632300Z PyGithub==2.3.0 2024-08-20T22:08:19.6632577Z Pygments==2.15.0 2024-08-20T22:08:19.6632857Z PyJWT==2.9.0 2024-08-20T22:08:19.6633123Z pylatexenc==2.10 2024-08-20T22:08:19.6633388Z pylint==3.2.6 2024-08-20T22:08:19.6633651Z PyNaCl==1.5.0 2024-08-20T22:08:19.6633923Z pynvml==11.5.3 2024-08-20T22:08:19.6634186Z pyparsing==3.1.2 2024-08-20T22:08:19.6634472Z pypdfium2==4.30.0 2024-08-20T22:08:19.6634751Z pysbd==0.3.4 2024-08-20T22:08:19.6635019Z PySocks==1.7.1 2024-08-20T22:08:19.6635275Z pytest==7.3.2 2024-08-20T22:08:19.6635595Z pytest-benchmark==4.0.0 2024-08-20T22:08:19.6635948Z pytest-cpp==2.3.0 2024-08-20T22:08:19.6636289Z pytest-flakefinder==1.1.0 2024-08-20T22:08:19.6636663Z pytest-rerunfailures==14.0 2024-08-20T22:08:19.6637035Z pytest-xdist==3.3.1 2024-08-20T22:08:19.6637386Z python-dateutil==2.9.0.post0 2024-08-20T22:08:19.6638172Z python-doctr @ git+https://github.com/xuzhao9/doctr.git@7f8a3e61bd6480c4e7b7cde47af0aa2153309c56 2024-08-20T22:08:19.6638935Z python-etcd==0.4.5 2024-08-20T22:08:19.6640005Z pytorch-labs-segment-anything-fast @ git+https://github.com/pytorch-labs/segment-anything-fast.git@de861af9badd03e8b40f5e063d70754a3dc6b4f4 2024-08-20T22:08:19.6641709Z -e git+https://github.com/pytorch/pytorch_sphinx_theme.git@05ee809ecf4c396cce8f44a172484f2946eb255a#egg=pytorch_sphinx_theme 2024-08-20T22:08:19.6642581Z pytz==2024.1 2024-08-20T22:08:19.6642857Z PyWavelets==1.4.1 2024-08-20T22:08:19.6643316Z PyYAML @ file:///croot/pyyaml_1698096049011/work 2024-08-20T22:08:19.6643758Z pyzmq==26.1.1 2024-08-20T22:08:19.6644018Z rapidfuzz==3.9.6 2024-08-20T22:08:19.6644302Z rdflib==7.0.0 2024-08-20T22:08:19.6644573Z redis==5.0.8 2024-08-20T22:08:19.6644845Z referencing==0.35.1 2024-08-20T22:08:19.6645131Z regex==2024.7.24 2024-08-20T22:08:19.6645409Z requests==2.32.3 2024-08-20T22:08:19.6645751Z requests-oauthlib==2.0.0 2024-08-20T22:08:19.6646064Z resampy==0.3.1 2024-08-20T22:08:19.6646343Z rfc3986==1.5.0 2024-08-20T22:08:19.6646609Z rich==13.7.1 2024-08-20T22:08:19.6646873Z rockset==1.0.3 2024-08-20T22:08:19.6647165Z rpds-py==0.20.0 2024-08-20T22:08:19.6647438Z rsa==4.9 2024-08-20T22:08:19.6647691Z s3transfer==0.5.2 2024-08-20T22:08:19.6647968Z safetensors==0.4.4 2024-08-20T22:08:19.6648302Z scikit-image==0.19.3 2024-08-20T22:08:19.6648731Z scikit-learn==1.3.2 2024-08-20T22:08:19.6649022Z scipy==1.10.1 2024-08-20T22:08:19.6649279Z segments==2.2.1 2024-08-20T22:08:19.6649566Z sentencepiece==0.2.0 2024-08-20T22:08:19.6649904Z sentry-sdk==2.13.0 2024-08-20T22:08:19.6650183Z setproctitle==1.3.3 2024-08-20T22:08:19.6650478Z shapely==2.0.6 2024-08-20T22:08:19.6650756Z shellingham==1.5.4 2024-08-20T22:08:19.6651049Z simplejson==3.19.3 2024-08-20T22:08:19.6651415Z six @ file:///tmp/build/80754af9/six_1644875935023/work 2024-08-20T22:08:19.6651910Z smart-open==7.0.4 2024-08-20T22:08:19.6652199Z smmap==5.0.1 2024-08-20T22:08:19.6652463Z snowballstemmer==2.2.0 2024-08-20T22:08:19.6652784Z sortedcontainers==2.4.0 2024-08-20T22:08:19.6653099Z soundfile==0.12.1 2024-08-20T22:08:19.6653385Z soupsieve==2.6 2024-08-20T22:08:19.6653646Z spacy==3.7.6 2024-08-20T22:08:19.6653947Z spacy-legacy==3.0.12 2024-08-20T22:08:19.6654284Z spacy-loggers==1.0.5 2024-08-20T22:08:19.6654570Z Sphinx==5.3.0 2024-08-20T22:08:19.6654888Z sphinx-copybutton==0.5.0 2024-08-20T22:08:19.6655255Z sphinx-panels==0.4.1 2024-08-20T22:08:19.6655632Z sphinxcontrib-applehelp==1.0.4 2024-08-20T22:08:19.6656044Z sphinxcontrib-devhelp==1.0.2 2024-08-20T22:08:19.6656460Z sphinxcontrib-htmlhelp==2.0.1 2024-08-20T22:08:19.6656871Z sphinxcontrib-jsmath==1.0.1 2024-08-20T22:08:19.6657268Z sphinxcontrib-katex==0.8.6 2024-08-20T22:08:19.6657646Z sphinxcontrib-qthelp==1.0.3 2024-08-20T22:08:19.6658079Z sphinxcontrib-serializinghtml==1.1.5 2024-08-20T22:08:19.6658482Z SQLAlchemy==2.0.32 2024-08-20T22:08:19.6658754Z srsly==2.4.8 2024-08-20T22:08:19.6659051Z stack-data==0.6.3 2024-08-20T22:08:19.6659337Z stempeg==0.2.3 2024-08-20T22:08:19.6659608Z submitit==1.5.1 2024-08-20T22:08:19.6659867Z sympy==1.12.1 2024-08-20T22:08:19.6660131Z tabulate==0.9.0 2024-08-20T22:08:19.6660451Z tb-nightly==2.13.0a20230426 2024-08-20T22:08:19.6660774Z tensorboard==2.13.0 2024-08-20T22:08:19.6661132Z tensorboard-data-server==0.7.2 2024-08-20T22:08:19.6661498Z tensorboardX==2.6.2.2 2024-08-20T22:08:19.6661807Z termcolor==2.4.0 2024-08-20T22:08:19.6662075Z thinc==8.2.5 2024-08-20T22:08:19.6662349Z threadpoolctl==3.5.0 2024-08-20T22:08:19.6662649Z tifffile==2023.7.10 2024-08-20T22:08:19.6663474Z timm @ git+https://github.com/huggingface/pytorch-image-models.git@730b907b4d45a4713cbc425cbf224c46089fd514 2024-08-20T22:08:19.6664457Z tlparse==0.3.25 2024-08-20T22:08:19.6664744Z tokenizers==0.15.2 2024-08-20T22:08:19.6665029Z tomli==2.0.1 2024-08-20T22:08:19.6665310Z tomlkit==0.13.2 2024-08-20T22:08:19.6666583Z torch @ file:///var/lib/jenkins/workspace/dist/torch-2.5.0a0%2Bgit40ec5f6-cp38-cp38-linux_x86_64.whl#sha256=af3cdc81a563c847a86d810c93c14c6dabe8cf56a0cb55c23bba116e1c3deca6 2024-08-20T22:08:19.6668301Z torch_geometric @ git+https://github.com/pyg-team/pytorch_geometric.git@cabcd4097442ba60aa1efa11e1619dd9bb8fb527 2024-08-20T22:08:19.6669404Z torchao @ git+https://github.com/pytorch/ao.git@2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T22:08:19.6670401Z torchaudio @ git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:08:19.6671428Z torchdata @ git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:08:19.6672711Z torchmultimodal @ git+https://github.com/facebookresearch/multimodal.git@e4d288b45b89cee462a21ab264405f3f368adc21 2024-08-20T22:08:19.6673860Z torchtext @ git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:08:19.6674889Z torchvision @ git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:08:19.6675587Z tornado==6.4.1 2024-08-20T22:08:19.6675857Z tqdm==4.66.5 2024-08-20T22:08:19.6676126Z traitlets==5.14.3 2024-08-20T22:08:19.6676422Z transformers==4.38.1 2024-08-20T22:08:19.6676819Z treetable==0.2.5 2024-08-20T22:08:19.6677627Z -e git+https://github.com/openai/triton@dedb7bdf339a3546896d4820366ca562c586bfa0#egg=triton&subdirectory=python 2024-08-20T22:08:19.6678411Z typeguard==4.3.0 2024-08-20T22:08:19.6678682Z typer==0.12.4 2024-08-20T22:08:19.6678974Z typing-inspect==0.9.0 2024-08-20T22:08:19.6679366Z typing_extensions==4.12.2 2024-08-20T22:08:19.6679682Z tzdata==2024.1 2024-08-20T22:08:19.6679934Z Unidecode==1.3.8 2024-08-20T22:08:19.6680282Z unittest-xml-reporting==3.2.0 2024-08-20T22:08:19.6680631Z uritemplate==4.1.1 2024-08-20T22:08:19.6680910Z urllib3==1.26.19 2024-08-20T22:08:19.6681166Z visdom==0.2.4 2024-08-20T22:08:19.6681421Z wandb==0.17.7 2024-08-20T22:08:19.6681735Z wasabi==1.1.3 2024-08-20T22:08:19.6681985Z wcwidth==0.2.13 2024-08-20T22:08:19.6682427Z weasel==0.4.1 2024-08-20T22:08:19.6682747Z websocket-client==1.8.0 2024-08-20T22:08:19.6683069Z Werkzeug==3.0.3 2024-08-20T22:08:19.6683330Z wrapt==1.16.0 2024-08-20T22:08:19.6683598Z xdoctest==1.1.0 2024-08-20T22:08:19.6683871Z xxhash==3.5.0 2024-08-20T22:08:19.6684135Z xyzservices==2024.6.0 2024-08-20T22:08:19.6684440Z yacs==0.1.8 2024-08-20T22:08:19.6684694Z yarl==1.9.4 2024-08-20T22:08:19.6684988Z z3-solver==4.12.2.0 2024-08-20T22:08:19.6685265Z zipp==3.20.0 2024-08-20T22:08:19.6880936Z + popd 2024-08-20T22:08:19.6881248Z ~/workspace 2024-08-20T22:08:19.6881922Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing != *cpu* ]] 2024-08-20T22:08:19.6882432Z ++ pwd 2024-08-20T22:08:19.6883736Z + PYTHONPATH=/var/lib/jenkins/workspace/torchbench 2024-08-20T22:08:19.6884238Z + test_dynamo_benchmark torchbench 0 2024-08-20T22:08:19.6887612Z ++ pwd 2024-08-20T22:08:19.6889709Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:08:19.6890320Z + local suite=torchbench 2024-08-20T22:08:19.6890646Z + shift 2024-08-20T22:08:19.6890885Z + local shard_id=0 2024-08-20T22:08:19.6891167Z + shift 2024-08-20T22:08:19.6891583Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *perf_compare* ]] 2024-08-20T22:08:19.6892253Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *perf* ]] 2024-08-20T22:08:19.6892862Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *cpu* ]] 2024-08-20T22:08:19.6893437Z + local dt=float32 2024-08-20T22:08:19.6893850Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *amp* ]] 2024-08-20T22:08:19.6894497Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *freezing* ]] 2024-08-20T22:08:19.6895448Z + test_single_dynamo_benchmark inference torchbench 0 --inference --float32 --freezing 2024-08-20T22:08:19.6896078Z ++ pwd 2024-08-20T22:08:19.6896531Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:08:19.6897202Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:08:19.6911559Z + local name=inference 2024-08-20T22:08:19.6911907Z + shift 2024-08-20T22:08:19.6912187Z + local suite=torchbench 2024-08-20T22:08:19.6912850Z + shift 2024-08-20T22:08:19.6913312Z + local shard_id=0 2024-08-20T22:08:19.6913696Z + shift 2024-08-20T22:08:19.6913951Z + partition_flags=() 2024-08-20T22:08:19.6914276Z + local partition_flags 2024-08-20T22:08:19.6914660Z + [[ -n 2 ]] 2024-08-20T22:08:19.6914952Z + [[ -n 0 ]] 2024-08-20T22:08:19.6915559Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T22:08:19.6916370Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *perf_compare* ]] 2024-08-20T22:08:19.6917024Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *perf* ]] 2024-08-20T22:08:19.6917856Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *aot_inductor* ]] 2024-08-20T22:08:19.6918582Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing != *cpu_aot_inductor* ]] 2024-08-20T22:08:19.6919272Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *_avx2* ]] 2024-08-20T22:08:19.6919900Z + [[ dynamic_cpu_aot_inductor_torchbench_freezing == *_avx512* ]] 2024-08-20T22:08:19.6922271Z + python benchmarks/dynamo/torchbench.py --ci --accuracy --timing --explain --export-aot-inductor --dynamic-shapes --dynamic-batch-only --device cpu --inference --float32 --freezing --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv 2024-08-20T22:08:24.7764664Z 2024-08-20T22:08:26.6211099Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:08:26.6211582Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:08:26.6238403Z cpu eval BERT_pytorch 2024-08-20T22:08:27.0366298Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:27.2226304Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:27.3989626Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:27.9164073Z ERROR:common: 2024-08-20T22:08:27.9164478Z Traceback (most recent call last): 2024-08-20T22:08:27.9165233Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:08:27.9166034Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:08:27.9166838Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:08:27.9167792Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:08:27.9168678Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:08:27.9169415Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:08:27.9170585Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:08:27.9171466Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:08:27.9172401Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:08:27.9173218Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:08:27.9174234Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:08:27.9175122Z return self._call_impl(*args, **kwargs) 2024-08-20T22:08:27.9176077Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:08:27.9176918Z return forward_call(*args, **kwargs) 2024-08-20T22:08:27.9177802Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:08:27.9178556Z return fn(*args, **kwargs) 2024-08-20T22:08:27.9179502Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:08:27.9180406Z return self._call_impl(*args, **kwargs) 2024-08-20T22:08:27.9181360Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:08:27.9182200Z return forward_call(*args, **kwargs) 2024-08-20T22:08:27.9183120Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:08:27.9183972Z return self._torchdynamo_orig_callable( 2024-08-20T22:08:27.9184919Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:08:27.9185730Z return _compile( 2024-08-20T22:08:27.9187329Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:08:27.9188309Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:08:27.9189447Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:08:27.9190395Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:08:27.9191431Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:08:27.9192256Z return function(*args, **kwargs) 2024-08-20T22:08:27.9193221Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:08:27.9194140Z out_code = transform_code_object(code, transform) 2024-08-20T22:08:27.9195298Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:08:27.9196468Z transformations(instructions, code_options) 2024-08-20T22:08:27.9197398Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:08:27.9198194Z return fn(*args, **kwargs) 2024-08-20T22:08:27.9199092Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:08:27.9199907Z tracer.run() 2024-08-20T22:08:27.9200703Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:08:27.9201495Z super().run() 2024-08-20T22:08:27.9202409Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:08:27.9203214Z while self.step(): 2024-08-20T22:08:27.9204036Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:08:27.9204906Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:08:27.9205898Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:08:27.9206736Z return inner_fn(self, inst) 2024-08-20T22:08:27.9207675Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:08:27.9208551Z self.call_function(fn, args, {}) 2024-08-20T22:08:27.9209521Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:08:27.9210650Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:08:27.9211831Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:08:27.9212764Z return super().call_function(tx, args, kwargs) 2024-08-20T22:08:27.9213823Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:08:27.9214759Z return super().call_function(tx, args, kwargs) 2024-08-20T22:08:27.9215807Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:08:27.9216887Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:08:27.9218157Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:08:27.9219306Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:08:27.9220488Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:08:27.9221411Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:08:27.9222463Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:08:27.9223682Z tracer.run() 2024-08-20T22:08:27.9224509Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:08:27.9225293Z while self.step(): 2024-08-20T22:08:27.9226138Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:08:27.9227006Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:08:27.9227990Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:08:27.9228809Z return inner_fn(self, inst) 2024-08-20T22:08:27.9229754Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:08:27.9230706Z self.call_function(fn, args, {}) 2024-08-20T22:08:27.9231674Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:08:27.9232799Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:08:27.9233982Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:08:27.9234925Z return super().call_function(tx, args, kwargs) 2024-08-20T22:08:27.9235968Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:08:27.9236898Z return super().call_function(tx, args, kwargs) 2024-08-20T22:08:27.9237930Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:08:27.9239023Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:08:27.9240302Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:08:27.9241456Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:08:27.9242708Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:08:27.9243640Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:08:27.9244691Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:08:27.9245540Z tracer.run() 2024-08-20T22:08:27.9246344Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:08:27.9247127Z while self.step(): 2024-08-20T22:08:27.9247961Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:08:27.9248834Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:08:27.9249845Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1739, in STORE_ATTR 2024-08-20T22:08:27.9250658Z assert ( 2024-08-20T22:08:27.9251072Z AssertionError: Mutating module attribute mask during export. 2024-08-20T22:08:27.9251493Z 2024-08-20T22:08:27.9251595Z from user code: 2024-08-20T22:08:27.9252466Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch/bert_pytorch/model/bert.py", line 50, in forward 2024-08-20T22:08:27.9253442Z x = transformer.forward(x, mask) 2024-08-20T22:08:27.9254467Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch/bert_pytorch/model/transformer.py", line 47, in forward 2024-08-20T22:08:27.9255491Z self.lambda_module.set_mask(mask) 2024-08-20T22:08:27.9256508Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch/bert_pytorch/model/transformer.py", line 16, in set_mask 2024-08-20T22:08:27.9257503Z self.mask = mask 2024-08-20T22:08:27.9257705Z 2024-08-20T22:08:27.9258085Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:08:27.9258550Z 2024-08-20T22:08:27.9258839Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:08:27.9259397Z fail_to_run 2024-08-20T22:08:27.9260003Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:27.9260701Z TIMING: 2024-08-20T22:08:27.9261465Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:12 | FakeTensorMode.__torch_dispatch__:53 | attempt fast:2 | fast is_contiguous:2 2024-08-20T22:08:27.9262532Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:08:30.9137024Z 2024-08-20T22:08:33.9325008Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:08:33.9325477Z loading model: 0it [00:03, ?it/s] 2024-08-20T22:08:33.9340518Z cpu eval Background_Matting 2024-08-20T22:08:33.9345122Z pass_due_to_skip 2024-08-20T22:08:33.9346311Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:33.9347032Z TIMING: 2024-08-20T22:08:33.9347300Z STATS: call_* op count: 0 2024-08-20T22:08:33.9347831Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:08:37.1321754Z 2024-08-20T22:08:39.2370468Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:08:39.2370955Z loading model: 0it [00:02, ?it/s] 2024-08-20T22:08:39.2377273Z cpu eval LearningToPaint 2024-08-20T22:08:39.3293667Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:39.3599828Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:39.3876503Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:09:09.5429589Z pass 2024-08-20T22:09:09.5431443Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:09:09.5433528Z TIMING: backend_compile:0.0001 entire_frame_compile:1.10077 total_wall_time:1.10077 2024-08-20T22:09:09.5435640Z STATS: call_* op count: 72 | FakeTensor.__torch_dispatch__:356 | FakeTensorMode.__torch_dispatch__:6864 | attempt fast:244 | fast is_contiguous:212 | ProxyTorchDispatchMode.__torch_dispatch__:961 | slow no contiguity match:32 2024-08-20T22:09:09.5437672Z Dynamo produced 1 graphs covering 72 ops with 0 graph breaks (0 unique) 2024-08-20T22:09:13.0991076Z 2024-08-20T22:09:14.3765572Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/vgg16-397923af.pth 2024-08-20T22:09:14.3958388Z 2024-08-20T22:09:14.3958645Z 2024-08-20T22:09:14.4963373Z 0% 0.00/528M [00:00 2024-08-20T22:18:50.7357352Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:18:50.7358338Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:18:50.7359226Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:18:50.7360210Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:18:50.7360996Z return node.target(*args, **kwargs) 2024-08-20T22:18:50.7362059Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:18:50.7363006Z r = self.shape_env.evaluate_expr( 2024-08-20T22:18:50.7364216Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:18:50.7365077Z return retlog(event.run(self)) 2024-08-20T22:18:50.7366010Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:18:50.7366852Z return self.f(*args, **kwargs) 2024-08-20T22:18:50.7367885Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:18:50.7369188Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:18:50.7370633Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:18:50.7371600Z raise self._make_data_dependent_error( 2024-08-20T22:18:50.7372877Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:18:50.7374410Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:18:50.7375041Z 2024-08-20T22:18:50.7375316Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:18:50.7376353Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:18:50.7377254Z return node.target(*args, **kwargs) 2024-08-20T22:18:50.7377536Z 2024-08-20T22:18:50.7377734Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:18:50.7378618Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:18:50.7379552Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:18:50.7380741Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:18:50.7381556Z 2024-08-20T22:18:50.7381686Z User Stack (most recent call last): 2024-08-20T22:18:50.7382099Z (snipped, see stack below for prefix) 2024-08-20T22:18:50.7383057Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:18:50.7383922Z return self.inference(batched_inputs) 2024-08-20T22:18:50.7384894Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:18:50.7385873Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:18:50.7386927Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:18:50.7387830Z images = ImageList.from_tensors( 2024-08-20T22:18:50.7388798Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:18:50.7389860Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:18:50.7390259Z 2024-08-20T22:18:50.7390513Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:18:50.7390905Z 2024-08-20T22:18:50.7391179Z During handling of the above exception, another exception occurred: 2024-08-20T22:18:50.7391624Z 2024-08-20T22:18:50.7391752Z Traceback (most recent call last): 2024-08-20T22:18:50.7392451Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:18:50.7393200Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:18:50.7393964Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:18:50.7394882Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:18:50.7395715Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:18:50.7396419Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:18:50.7397388Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:18:50.7398249Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:18:50.7399152Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:18:50.7399946Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:18:50.7400915Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:18:50.7401795Z return self._call_impl(*args, **kwargs) 2024-08-20T22:18:50.7402729Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:18:50.7403544Z return forward_call(*args, **kwargs) 2024-08-20T22:18:50.7404468Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:18:50.7405388Z return fn(*args, **kwargs) 2024-08-20T22:18:50.7406341Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:18:50.7407254Z return self._call_impl(*args, **kwargs) 2024-08-20T22:18:50.7408208Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:18:50.7409037Z return forward_call(*args, **kwargs) 2024-08-20T22:18:50.7409978Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:18:50.7410836Z return self._torchdynamo_orig_callable( 2024-08-20T22:18:50.7411867Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:18:50.7412662Z return _compile( 2024-08-20T22:18:50.7413501Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:18:50.7414459Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:18:50.7415560Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:18:50.7416510Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:18:50.7417631Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:18:50.7418447Z return function(*args, **kwargs) 2024-08-20T22:18:50.7419436Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:18:50.7420345Z out_code = transform_code_object(code, transform) 2024-08-20T22:18:50.7421454Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:18:50.7422444Z transformations(instructions, code_options) 2024-08-20T22:18:50.7423410Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:18:50.7424187Z return fn(*args, **kwargs) 2024-08-20T22:18:50.7425052Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:18:50.7425843Z tracer.run() 2024-08-20T22:18:50.7426627Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:18:50.7427412Z super().run() 2024-08-20T22:18:50.7428198Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:18:50.7428980Z while self.step(): 2024-08-20T22:18:50.7429776Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:18:50.7430619Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:18:50.7431662Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:18:50.7432436Z return inner_fn(self, inst) 2024-08-20T22:18:50.7433303Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:18:50.7434119Z self.call_function(fn, args, {}) 2024-08-20T22:18:50.7435021Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:18:50.7436076Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:18:50.7437183Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:18:50.7438045Z return super().call_function(tx, args, kwargs) 2024-08-20T22:18:50.7439028Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:18:50.7439897Z return super().call_function(tx, args, kwargs) 2024-08-20T22:18:50.7440865Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:18:50.7441856Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:18:50.7443044Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:18:50.7444181Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:18:50.7445615Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:18:50.7446543Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:18:50.7447602Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:18:50.7448451Z tracer.run() 2024-08-20T22:18:50.7449259Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:18:50.7450064Z while self.step(): 2024-08-20T22:18:50.7450892Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:18:50.7451761Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:18:50.7452743Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:18:50.7453628Z return inner_fn(self, inst) 2024-08-20T22:18:50.7454573Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:18:50.7455455Z self.call_function(fn, args, {}) 2024-08-20T22:18:50.7456422Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:18:50.7457640Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:18:50.7458796Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:18:50.7459718Z return super().call_function(tx, args, kwargs) 2024-08-20T22:18:50.7460916Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:18:50.7461945Z return super().call_function(tx, args, kwargs) 2024-08-20T22:18:50.7462955Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:18:50.7464312Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:18:50.7465599Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:18:50.7466757Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:18:50.7467945Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:18:50.7468875Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:18:50.7469924Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:18:50.7470766Z tracer.run() 2024-08-20T22:18:50.7471574Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:18:50.7472360Z while self.step(): 2024-08-20T22:18:50.7473203Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:18:50.7474078Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:18:50.7475066Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:18:50.7476023Z return inner_fn(self, inst) 2024-08-20T22:18:50.7477286Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:18:50.7478306Z self.call_function(fn, args, kwargs) 2024-08-20T22:18:50.7479294Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:18:50.7480421Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:18:50.7481734Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:18:50.7482686Z return super().call_function(tx, args, kwargs) 2024-08-20T22:18:50.7483738Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:18:50.7484904Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:18:50.7486185Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:18:50.7487332Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:18:50.7488524Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:18:50.7489465Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:18:50.7490602Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:18:50.7491438Z tracer.run() 2024-08-20T22:18:50.7492249Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:18:50.7493057Z while self.step(): 2024-08-20T22:18:50.7493894Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:18:50.7494993Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:18:50.7496197Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:18:50.7497207Z return inner_fn(self, inst) 2024-08-20T22:18:50.7498076Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:18:50.7498975Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:18:50.7500053Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:18:50.7500916Z return handler(tx, args, kwargs) 2024-08-20T22:18:50.7501955Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:18:50.7502890Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:18:50.7503836Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:18:50.7504814Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:18:50.7505931Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:18:50.7506985Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:18:50.7508047Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:18:50.7508848Z raise UserError( # noqa: B904 2024-08-20T22:18:50.7511501Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:18:50.7513787Z 2024-08-20T22:18:50.7514052Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:18:50.7515134Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:18:50.7515913Z return node.target(*args, **kwargs) 2024-08-20T22:18:50.7516213Z 2024-08-20T22:18:50.7516403Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:18:50.7517260Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:18:50.7518307Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:18:50.7519615Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:18:50.7520421Z 2024-08-20T22:18:50.7520549Z User Stack (most recent call last): 2024-08-20T22:18:50.7520963Z (snipped, see stack below for prefix) 2024-08-20T22:18:50.7521937Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:18:50.7522816Z return self.inference(batched_inputs) 2024-08-20T22:18:50.7523790Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:18:50.7525019Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:18:50.7526212Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:18:50.7527147Z images = ImageList.from_tensors( 2024-08-20T22:18:50.7528134Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:18:50.7529229Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:18:50.7529655Z 2024-08-20T22:18:50.7529905Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:18:50.7531093Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:18:50.7531904Z 2024-08-20T22:18:50.7532023Z from user code: 2024-08-20T22:18:50.7532887Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:18:50.7533788Z return self.inference(batched_inputs) 2024-08-20T22:18:50.7534805Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:18:50.7535744Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:18:50.7536939Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:18:50.7537809Z images = ImageList.from_tensors( 2024-08-20T22:18:50.7538737Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:18:50.7539754Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:18:50.7540138Z 2024-08-20T22:18:50.7540420Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:18:50.7540836Z 2024-08-20T22:18:50.7541105Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:18:50.7541631Z fail_to_run 2024-08-20T22:18:50.7542203Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:18:50.7542859Z TIMING: 2024-08-20T22:18:50.7543376Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:40 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:18:50.7544197Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:18:53.9407456Z 2024-08-20T22:19:06.4383872Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:19:06.4384328Z loading model: 0it [00:12, ?it/s] 2024-08-20T22:19:06.4402748Z cpu eval detectron2_fasterrcnn_r_101_dc5 2024-08-20T22:19:24.6425589Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_101_dc5. Setting accuracy check to cosine 2024-08-20T22:19:24.6562063Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:19:37.3319307Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:19:49.9595007Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:19:56.8882324Z W0820 22:19:56.887288 15705 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:19:56.8884747Z E0820 22:19:56.887454 15705 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:19:56.9137122Z ERROR:common: 2024-08-20T22:19:56.9137474Z Traceback (most recent call last): 2024-08-20T22:19:56.9138457Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:19:56.9139271Z return node.target(*args, **kwargs) 2024-08-20T22:19:56.9147191Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:19:56.9148400Z r = self.shape_env.evaluate_expr( 2024-08-20T22:19:56.9149405Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:19:56.9150269Z return retlog(event.run(self)) 2024-08-20T22:19:56.9151197Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:19:56.9152036Z return self.f(*args, **kwargs) 2024-08-20T22:19:56.9153074Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:19:56.9154398Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:19:56.9155825Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:19:56.9156803Z raise self._make_data_dependent_error( 2024-08-20T22:19:56.9158150Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:19:56.9159193Z 2024-08-20T22:19:56.9159474Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:19:56.9160518Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:19:56.9161312Z return node.target(*args, **kwargs) 2024-08-20T22:19:56.9161611Z 2024-08-20T22:19:56.9161805Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:19:56.9162598Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:19:56.9163677Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:56.9165071Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:19:56.9165919Z 2024-08-20T22:19:56.9166063Z User Stack (most recent call last): 2024-08-20T22:19:56.9166482Z (snipped, see stack below for prefix) 2024-08-20T22:19:56.9167486Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:19:56.9168365Z return self.inference(batched_inputs) 2024-08-20T22:19:56.9169369Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:19:56.9170307Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:19:56.9171403Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:19:56.9172330Z images = ImageList.from_tensors( 2024-08-20T22:19:56.9173314Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:19:56.9174404Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:19:56.9175046Z 2024-08-20T22:19:56.9175315Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:56.9175737Z 2024-08-20T22:19:56.9176129Z The above exception was the direct cause of the following exception: 2024-08-20T22:19:56.9176576Z 2024-08-20T22:19:56.9176703Z Traceback (most recent call last): 2024-08-20T22:19:56.9177593Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:19:56.9178394Z ret_val = wrap_fake_exception( 2024-08-20T22:19:56.9179285Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:19:56.9180090Z return fn() 2024-08-20T22:19:56.9180835Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:19:56.9181798Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:19:56.9182751Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:19:56.9183610Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:19:56.9184566Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:19:56.9185339Z return node.target(*args, **kwargs) 2024-08-20T22:19:56.9186343Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:19:56.9187255Z r = self.shape_env.evaluate_expr( 2024-08-20T22:19:56.9188192Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:19:56.9189033Z return retlog(event.run(self)) 2024-08-20T22:19:56.9189919Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:19:56.9190735Z return self.f(*args, **kwargs) 2024-08-20T22:19:56.9191739Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:19:56.9193009Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:19:56.9194400Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:19:56.9195326Z raise self._make_data_dependent_error( 2024-08-20T22:19:56.9196561Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:19:56.9198046Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:19:56.9198667Z 2024-08-20T22:19:56.9198941Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:19:56.9199934Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:19:56.9200710Z return node.target(*args, **kwargs) 2024-08-20T22:19:56.9201002Z 2024-08-20T22:19:56.9201186Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:19:56.9201954Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:19:56.9202979Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:56.9204387Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:19:56.9205222Z 2024-08-20T22:19:56.9205354Z User Stack (most recent call last): 2024-08-20T22:19:56.9205774Z (snipped, see stack below for prefix) 2024-08-20T22:19:56.9206769Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:19:56.9207721Z return self.inference(batched_inputs) 2024-08-20T22:19:56.9208734Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:19:56.9209672Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:19:56.9210774Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:19:56.9211692Z images = ImageList.from_tensors( 2024-08-20T22:19:56.9212693Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:19:56.9213782Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:19:56.9214195Z 2024-08-20T22:19:56.9214455Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:56.9214976Z 2024-08-20T22:19:56.9215380Z During handling of the above exception, another exception occurred: 2024-08-20T22:19:56.9215790Z 2024-08-20T22:19:56.9215914Z Traceback (most recent call last): 2024-08-20T22:19:56.9216583Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:19:56.9217289Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:19:56.9218030Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:19:56.9218894Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:19:56.9219685Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:19:56.9220360Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:19:56.9221305Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:19:56.9222118Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:19:56.9222978Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:19:56.9223728Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:19:56.9224666Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:19:56.9225503Z return self._call_impl(*args, **kwargs) 2024-08-20T22:19:56.9226560Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:19:56.9227385Z return forward_call(*args, **kwargs) 2024-08-20T22:19:56.9228241Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:19:56.9228992Z return fn(*args, **kwargs) 2024-08-20T22:19:56.9229902Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:19:56.9230785Z return self._call_impl(*args, **kwargs) 2024-08-20T22:19:56.9231714Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:19:56.9232538Z return forward_call(*args, **kwargs) 2024-08-20T22:19:56.9233442Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:19:56.9234271Z return self._torchdynamo_orig_callable( 2024-08-20T22:19:56.9235199Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:19:56.9235978Z return _compile( 2024-08-20T22:19:56.9236777Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:19:56.9237712Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:19:56.9238777Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:19:56.9239774Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:19:56.9240792Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:19:56.9241574Z return function(*args, **kwargs) 2024-08-20T22:19:56.9242714Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:19:56.9243883Z out_code = transform_code_object(code, transform) 2024-08-20T22:19:56.9245041Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:19:56.9246036Z transformations(instructions, code_options) 2024-08-20T22:19:56.9246976Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:19:56.9247851Z return fn(*args, **kwargs) 2024-08-20T22:19:56.9248739Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:19:56.9249541Z tracer.run() 2024-08-20T22:19:56.9250354Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:19:56.9251149Z super().run() 2024-08-20T22:19:56.9251953Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:19:56.9252736Z while self.step(): 2024-08-20T22:19:56.9253564Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:19:56.9254426Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:19:56.9255487Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:19:56.9256245Z return inner_fn(self, inst) 2024-08-20T22:19:56.9257128Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:19:56.9257942Z self.call_function(fn, args, {}) 2024-08-20T22:19:56.9258839Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:19:56.9259874Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:19:56.9260971Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:19:56.9261842Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:56.9262812Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:19:56.9263678Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:56.9264971Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:19:56.9266068Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:19:56.9267341Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:19:56.9268499Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:19:56.9269671Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:19:56.9270610Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:19:56.9271662Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:19:56.9272511Z tracer.run() 2024-08-20T22:19:56.9273306Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:19:56.9274106Z while self.step(): 2024-08-20T22:19:56.9275053Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:19:56.9275925Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:19:56.9277098Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:19:56.9277856Z return inner_fn(self, inst) 2024-08-20T22:19:56.9278915Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:19:56.9279772Z self.call_function(fn, args, {}) 2024-08-20T22:19:56.9280711Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:19:56.9281797Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:19:56.9283018Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:19:56.9284224Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:56.9285281Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:19:56.9286201Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:56.9287248Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:19:56.9288342Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:19:56.9289619Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:19:56.9290776Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:19:56.9291949Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:19:56.9292888Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:19:56.9293944Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:19:56.9294784Z tracer.run() 2024-08-20T22:19:56.9295578Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:19:56.9296370Z while self.step(): 2024-08-20T22:19:56.9297206Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:19:56.9298069Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:19:56.9299036Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:19:56.9299864Z return inner_fn(self, inst) 2024-08-20T22:19:56.9300833Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:19:56.9301850Z self.call_function(fn, args, kwargs) 2024-08-20T22:19:56.9302797Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:19:56.9303888Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:19:56.9305046Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:19:56.9306061Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:56.9307032Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:19:56.9308027Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:19:56.9309205Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:19:56.9310271Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:19:56.9311426Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:19:56.9312278Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:19:56.9313257Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:19:56.9314049Z tracer.run() 2024-08-20T22:19:56.9314794Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:19:56.9315523Z while self.step(): 2024-08-20T22:19:56.9316296Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:19:56.9317094Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:19:56.9318057Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:19:56.9318821Z return inner_fn(self, inst) 2024-08-20T22:19:56.9319633Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:19:56.9320488Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:19:56.9321509Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:19:56.9322322Z return handler(tx, args, kwargs) 2024-08-20T22:19:56.9323569Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:19:56.9324525Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:19:56.9325510Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:19:56.9326520Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:19:56.9327655Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:19:56.9328733Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:19:56.9329829Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:19:56.9346029Z raise UserError( # noqa: B904 2024-08-20T22:19:56.9348879Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:19:56.9351149Z 2024-08-20T22:19:56.9351414Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:19:56.9352437Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:19:56.9353224Z return node.target(*args, **kwargs) 2024-08-20T22:19:56.9353508Z 2024-08-20T22:19:56.9353694Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:19:56.9354476Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:19:56.9355426Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:56.9356616Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:19:56.9357419Z 2024-08-20T22:19:56.9357562Z User Stack (most recent call last): 2024-08-20T22:19:56.9357964Z (snipped, see stack below for prefix) 2024-08-20T22:19:56.9358942Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:19:56.9359924Z return self.inference(batched_inputs) 2024-08-20T22:19:56.9360919Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:19:56.9361819Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:19:56.9362992Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:19:56.9364261Z images = ImageList.from_tensors( 2024-08-20T22:19:56.9365264Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:19:56.9366346Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:19:56.9366777Z 2024-08-20T22:19:56.9367027Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:56.9368366Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:19:56.9369178Z 2024-08-20T22:19:56.9369300Z from user code: 2024-08-20T22:19:56.9370180Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:19:56.9371067Z return self.inference(batched_inputs) 2024-08-20T22:19:56.9372068Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:19:56.9373005Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:19:56.9374094Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:19:56.9375014Z images = ImageList.from_tensors( 2024-08-20T22:19:56.9376121Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:19:56.9377188Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:19:56.9377589Z 2024-08-20T22:19:56.9377890Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:19:56.9378327Z 2024-08-20T22:19:56.9378611Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:19:56.9379155Z fail_to_run 2024-08-20T22:19:56.9379747Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:19:56.9380432Z TIMING: 2024-08-20T22:19:56.9380968Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:40 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:19:56.9381826Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:20:00.1173139Z 2024-08-20T22:20:08.7329577Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:20:08.7330044Z loading model: 0it [00:08, ?it/s] 2024-08-20T22:20:08.7352225Z cpu eval detectron2_fasterrcnn_r_101_fpn 2024-08-20T22:20:25.2831975Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_101_fpn. Setting accuracy check to cosine 2024-08-20T22:20:25.3088264Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:20:32.7636059Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:20:40.1684560Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:20:44.2886621Z W0820 22:20:44.287930 16032 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:20:44.2888924Z E0820 22:20:44.288093 16032 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:20:44.3151385Z ERROR:common: 2024-08-20T22:20:44.3151734Z Traceback (most recent call last): 2024-08-20T22:20:44.3153140Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:20:44.3158392Z return node.target(*args, **kwargs) 2024-08-20T22:20:44.3159519Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:20:44.3160468Z r = self.shape_env.evaluate_expr( 2024-08-20T22:20:44.3161444Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:20:44.3162316Z return retlog(event.run(self)) 2024-08-20T22:20:44.3163344Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:20:44.3164370Z return self.f(*args, **kwargs) 2024-08-20T22:20:44.3165433Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:20:44.3166929Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:20:44.3168376Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:20:44.3169350Z raise self._make_data_dependent_error( 2024-08-20T22:20:44.3170684Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:20:44.3171723Z 2024-08-20T22:20:44.3171995Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:20:44.3173025Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:20:44.3173822Z return node.target(*args, **kwargs) 2024-08-20T22:20:44.3174109Z 2024-08-20T22:20:44.3174315Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:20:44.3175101Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:20:44.3176068Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:20:44.3177284Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:20:44.3178111Z 2024-08-20T22:20:44.3178255Z User Stack (most recent call last): 2024-08-20T22:20:44.3178664Z (snipped, see stack below for prefix) 2024-08-20T22:20:44.3179657Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:20:44.3180543Z return self.inference(batched_inputs) 2024-08-20T22:20:44.3181541Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:20:44.3182464Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:20:44.3183649Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:20:44.3184579Z images = ImageList.from_tensors( 2024-08-20T22:20:44.3185557Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:20:44.3186650Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:20:44.3187062Z 2024-08-20T22:20:44.3187320Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:20:44.3187724Z 2024-08-20T22:20:44.3188026Z The above exception was the direct cause of the following exception: 2024-08-20T22:20:44.3188475Z 2024-08-20T22:20:44.3188607Z Traceback (most recent call last): 2024-08-20T22:20:44.3189513Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:20:44.3190336Z ret_val = wrap_fake_exception( 2024-08-20T22:20:44.3191369Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:20:44.3192189Z return fn() 2024-08-20T22:20:44.3192968Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:20:44.3193863Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:20:44.3194852Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:20:44.3195739Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:20:44.3196707Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:20:44.3197501Z return node.target(*args, **kwargs) 2024-08-20T22:20:44.3198546Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:20:44.3199551Z r = self.shape_env.evaluate_expr( 2024-08-20T22:20:44.3200505Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:20:44.3201369Z return retlog(event.run(self)) 2024-08-20T22:20:44.3202288Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:20:44.3203202Z return self.f(*args, **kwargs) 2024-08-20T22:20:44.3204222Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:20:44.3205536Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:20:44.3206975Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:20:44.3207947Z raise self._make_data_dependent_error( 2024-08-20T22:20:44.3209219Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:20:44.3210735Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:20:44.3211383Z 2024-08-20T22:20:44.3211653Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:20:44.3212677Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:20:44.3213470Z return node.target(*args, **kwargs) 2024-08-20T22:20:44.3213760Z 2024-08-20T22:20:44.3213946Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:20:44.3214732Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:20:44.3215696Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:20:44.3216917Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:20:44.3217739Z 2024-08-20T22:20:44.3217883Z User Stack (most recent call last): 2024-08-20T22:20:44.3218292Z (snipped, see stack below for prefix) 2024-08-20T22:20:44.3219283Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:20:44.3220170Z return self.inference(batched_inputs) 2024-08-20T22:20:44.3221169Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:20:44.3222087Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:20:44.3223187Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:20:44.3224114Z images = ImageList.from_tensors( 2024-08-20T22:20:44.3225110Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:20:44.3226264Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:20:44.3226692Z 2024-08-20T22:20:44.3226940Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:20:44.3227357Z 2024-08-20T22:20:44.3227642Z During handling of the above exception, another exception occurred: 2024-08-20T22:20:44.3228086Z 2024-08-20T22:20:44.3228235Z Traceback (most recent call last): 2024-08-20T22:20:44.3228961Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:20:44.3229712Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:20:44.3230511Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:20:44.3231461Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:20:44.3232369Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:20:44.3233086Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:20:44.3234090Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:20:44.3234972Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:20:44.3235896Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:20:44.3236699Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:20:44.3237707Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:20:44.3238609Z return self._call_impl(*args, **kwargs) 2024-08-20T22:20:44.3239556Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:20:44.3240388Z return forward_call(*args, **kwargs) 2024-08-20T22:20:44.3241262Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:20:44.3242039Z return fn(*args, **kwargs) 2024-08-20T22:20:44.3243051Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:20:44.3243956Z return self._call_impl(*args, **kwargs) 2024-08-20T22:20:44.3244904Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:20:44.3245755Z return forward_call(*args, **kwargs) 2024-08-20T22:20:44.3246695Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:20:44.3247560Z return self._torchdynamo_orig_callable( 2024-08-20T22:20:44.3248500Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:20:44.3249311Z return _compile( 2024-08-20T22:20:44.3250146Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:20:44.3251097Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:20:44.3252179Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:20:44.3253122Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:20:44.3254148Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:20:44.3254982Z return function(*args, **kwargs) 2024-08-20T22:20:44.3256015Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:20:44.3256913Z out_code = transform_code_object(code, transform) 2024-08-20T22:20:44.3258031Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:20:44.3259119Z transformations(instructions, code_options) 2024-08-20T22:20:44.3260025Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:20:44.3260798Z return fn(*args, **kwargs) 2024-08-20T22:20:44.3261670Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:20:44.3262463Z tracer.run() 2024-08-20T22:20:44.3263259Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:20:44.3264197Z super().run() 2024-08-20T22:20:44.3265183Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:20:44.3265985Z while self.step(): 2024-08-20T22:20:44.3266820Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:20:44.3267798Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:20:44.3268795Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:20:44.3269635Z return inner_fn(self, inst) 2024-08-20T22:20:44.3270589Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:20:44.3271452Z self.call_function(fn, args, {}) 2024-08-20T22:20:44.3272416Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:20:44.3273544Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:20:44.3274733Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:20:44.3275658Z return super().call_function(tx, args, kwargs) 2024-08-20T22:20:44.3276806Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:20:44.3277718Z return super().call_function(tx, args, kwargs) 2024-08-20T22:20:44.3278733Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:20:44.3279797Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:20:44.3281032Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:20:44.3282148Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:20:44.3283533Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:20:44.3284484Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:20:44.3285530Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:20:44.3286378Z tracer.run() 2024-08-20T22:20:44.3287190Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:20:44.3287985Z while self.step(): 2024-08-20T22:20:44.3288807Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:20:44.3289675Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:20:44.3290652Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:20:44.3291486Z return inner_fn(self, inst) 2024-08-20T22:20:44.3292424Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:20:44.3293305Z self.call_function(fn, args, {}) 2024-08-20T22:20:44.3294384Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:20:44.3295623Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:20:44.3296783Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:20:44.3297679Z return super().call_function(tx, args, kwargs) 2024-08-20T22:20:44.3298696Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:20:44.3299612Z return super().call_function(tx, args, kwargs) 2024-08-20T22:20:44.3300632Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:20:44.3301684Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:20:44.3302995Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:20:44.3304119Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:20:44.3305273Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:20:44.3306192Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:20:44.3307200Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:20:44.3308022Z tracer.run() 2024-08-20T22:20:44.3308801Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:20:44.3309580Z while self.step(): 2024-08-20T22:20:44.3310560Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:20:44.3311427Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:20:44.3312412Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:20:44.3313246Z return inner_fn(self, inst) 2024-08-20T22:20:44.3314206Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:20:44.3315107Z self.call_function(fn, args, kwargs) 2024-08-20T22:20:44.3316078Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:20:44.3317194Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:20:44.3318370Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:20:44.3319300Z return super().call_function(tx, args, kwargs) 2024-08-20T22:20:44.3320351Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:20:44.3321439Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:20:44.3322699Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:20:44.3323934Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:20:44.3325128Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:20:44.3326064Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:20:44.3327111Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:20:44.3327940Z tracer.run() 2024-08-20T22:20:44.3328749Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:20:44.3329549Z while self.step(): 2024-08-20T22:20:44.3330455Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:20:44.3331316Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:20:44.3332296Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:20:44.3333125Z return inner_fn(self, inst) 2024-08-20T22:20:44.3334010Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:20:44.3334926Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:20:44.3336117Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:20:44.3336971Z return handler(tx, args, kwargs) 2024-08-20T22:20:44.3338062Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:20:44.3338984Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:20:44.3339940Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:20:44.3340911Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:20:44.3342027Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:20:44.3343078Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:20:44.3344130Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:20:44.3344938Z raise UserError( # noqa: B904 2024-08-20T22:20:44.3347571Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:20:44.3349791Z 2024-08-20T22:20:44.3350058Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:20:44.3351062Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:20:44.3351843Z return node.target(*args, **kwargs) 2024-08-20T22:20:44.3352124Z 2024-08-20T22:20:44.3352307Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:20:44.3353083Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:20:44.3354032Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:20:44.3355229Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:20:44.3356029Z 2024-08-20T22:20:44.3356159Z User Stack (most recent call last): 2024-08-20T22:20:44.3356576Z (snipped, see stack below for prefix) 2024-08-20T22:20:44.3357546Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:20:44.3358422Z return self.inference(batched_inputs) 2024-08-20T22:20:44.3359391Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:20:44.3360297Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:20:44.3361369Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:20:44.3362281Z images = ImageList.from_tensors( 2024-08-20T22:20:44.3363498Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:20:44.3364934Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:20:44.3365364Z 2024-08-20T22:20:44.3365611Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:20:44.3366808Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:20:44.3367618Z 2024-08-20T22:20:44.3367738Z from user code: 2024-08-20T22:20:44.3368599Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:20:44.3369501Z return self.inference(batched_inputs) 2024-08-20T22:20:44.3370507Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:20:44.3371440Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:20:44.3372607Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:20:44.3373538Z images = ImageList.from_tensors( 2024-08-20T22:20:44.3374535Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:20:44.3375627Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:20:44.3376146Z 2024-08-20T22:20:44.3376443Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:20:44.3376882Z 2024-08-20T22:20:44.3377165Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:20:44.3377711Z fail_to_run 2024-08-20T22:20:44.3378298Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:20:44.3378982Z TIMING: 2024-08-20T22:20:44.3379522Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:43 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:20:44.3380384Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:20:47.5518126Z 2024-08-20T22:21:01.7557260Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:21:01.7557720Z loading model: 0it [00:14, ?it/s] 2024-08-20T22:21:01.7564296Z cpu eval detectron2_fasterrcnn_r_50_c4 2024-08-20T22:21:09.9176282Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_50_c4. Setting accuracy check to cosine 2024-08-20T22:21:09.9246710Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:21:31.8125394Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:21:53.8491351Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:22:05.2148527Z W0820 22:22:05.213959 16359 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:22:05.2150899Z E0820 22:22:05.214118 16359 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:22:05.2402383Z ERROR:common: 2024-08-20T22:22:05.2402731Z Traceback (most recent call last): 2024-08-20T22:22:05.2407173Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:05.2408022Z return node.target(*args, **kwargs) 2024-08-20T22:22:05.2409107Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:22:05.2410049Z r = self.shape_env.evaluate_expr( 2024-08-20T22:22:05.2411028Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:22:05.2411914Z return retlog(event.run(self)) 2024-08-20T22:22:05.2413056Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:22:05.2413900Z return self.f(*args, **kwargs) 2024-08-20T22:22:05.2414965Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:22:05.2416296Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:22:05.2417736Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:22:05.2418699Z raise self._make_data_dependent_error( 2024-08-20T22:22:05.2420036Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:22:05.2421209Z 2024-08-20T22:22:05.2421484Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:22:05.2422526Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:05.2423326Z return node.target(*args, **kwargs) 2024-08-20T22:22:05.2423616Z 2024-08-20T22:22:05.2423820Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:22:05.2424595Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:22:05.2425566Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:05.2426780Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:22:05.2427599Z 2024-08-20T22:22:05.2427743Z User Stack (most recent call last): 2024-08-20T22:22:05.2428165Z (snipped, see stack below for prefix) 2024-08-20T22:22:05.2429146Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:22:05.2430048Z return self.inference(batched_inputs) 2024-08-20T22:22:05.2431048Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:22:05.2431972Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:22:05.2433051Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:22:05.2433979Z images = ImageList.from_tensors( 2024-08-20T22:22:05.2434971Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:22:05.2436065Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:22:05.2436477Z 2024-08-20T22:22:05.2436724Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:05.2437147Z 2024-08-20T22:22:05.2437433Z The above exception was the direct cause of the following exception: 2024-08-20T22:22:05.2437891Z 2024-08-20T22:22:05.2438026Z Traceback (most recent call last): 2024-08-20T22:22:05.2438940Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:22:05.2439763Z ret_val = wrap_fake_exception( 2024-08-20T22:22:05.2440688Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:22:05.2441512Z return fn() 2024-08-20T22:22:05.2442279Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:22:05.2443256Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:22:05.2444232Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:22:05.2445119Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:22:05.2446189Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:05.2446984Z return node.target(*args, **kwargs) 2024-08-20T22:22:05.2448020Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:22:05.2449055Z r = self.shape_env.evaluate_expr( 2024-08-20T22:22:05.2450022Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:22:05.2450871Z return retlog(event.run(self)) 2024-08-20T22:22:05.2451795Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:22:05.2452636Z return self.f(*args, **kwargs) 2024-08-20T22:22:05.2453671Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:22:05.2455041Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:22:05.2456483Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:22:05.2457452Z raise self._make_data_dependent_error( 2024-08-20T22:22:05.2458720Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:22:05.2460243Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:22:05.2460880Z 2024-08-20T22:22:05.2461147Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:22:05.2462173Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:05.2462977Z return node.target(*args, **kwargs) 2024-08-20T22:22:05.2463264Z 2024-08-20T22:22:05.2463466Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:22:05.2464434Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:22:05.2465511Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:05.2466736Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:22:05.2467556Z 2024-08-20T22:22:05.2467702Z User Stack (most recent call last): 2024-08-20T22:22:05.2468113Z (snipped, see stack below for prefix) 2024-08-20T22:22:05.2469118Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:22:05.2470010Z return self.inference(batched_inputs) 2024-08-20T22:22:05.2471015Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:22:05.2471952Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:22:05.2473040Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:22:05.2473972Z images = ImageList.from_tensors( 2024-08-20T22:22:05.2474965Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:22:05.2476058Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:22:05.2476474Z 2024-08-20T22:22:05.2476813Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:05.2477199Z 2024-08-20T22:22:05.2477461Z During handling of the above exception, another exception occurred: 2024-08-20T22:22:05.2477882Z 2024-08-20T22:22:05.2478005Z Traceback (most recent call last): 2024-08-20T22:22:05.2478669Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:22:05.2479373Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:22:05.2480227Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:22:05.2481278Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:22:05.2482121Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:22:05.2483075Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:22:05.2484086Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:22:05.2484979Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:22:05.2485944Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:22:05.2486761Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:22:05.2487855Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:22:05.2488763Z return self._call_impl(*args, **kwargs) 2024-08-20T22:22:05.2489719Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:22:05.2490563Z return forward_call(*args, **kwargs) 2024-08-20T22:22:05.2491428Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:22:05.2492195Z return fn(*args, **kwargs) 2024-08-20T22:22:05.2493140Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:22:05.2494039Z return self._call_impl(*args, **kwargs) 2024-08-20T22:22:05.2494972Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:22:05.2495816Z return forward_call(*args, **kwargs) 2024-08-20T22:22:05.2496755Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:22:05.2497716Z return self._torchdynamo_orig_callable( 2024-08-20T22:22:05.2498623Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:22:05.2499405Z return _compile( 2024-08-20T22:22:05.2500214Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:22:05.2501141Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:22:05.2502203Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:22:05.2503188Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:22:05.2504334Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:22:05.2505148Z return function(*args, **kwargs) 2024-08-20T22:22:05.2506083Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:22:05.2506963Z out_code = transform_code_object(code, transform) 2024-08-20T22:22:05.2508081Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:22:05.2509057Z transformations(instructions, code_options) 2024-08-20T22:22:05.2509966Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:22:05.2510724Z return fn(*args, **kwargs) 2024-08-20T22:22:05.2511590Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:22:05.2512375Z tracer.run() 2024-08-20T22:22:05.2513163Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:22:05.2513928Z super().run() 2024-08-20T22:22:05.2514773Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:22:05.2515551Z while self.step(): 2024-08-20T22:22:05.2516366Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:22:05.2517192Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:22:05.2518222Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:22:05.2518993Z return inner_fn(self, inst) 2024-08-20T22:22:05.2519875Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:22:05.2520686Z self.call_function(fn, args, {}) 2024-08-20T22:22:05.2521571Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:22:05.2522676Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:22:05.2524072Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:22:05.2525005Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:05.2526047Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:22:05.2526985Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:05.2528032Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:22:05.2529121Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:22:05.2530382Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:22:05.2531540Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:22:05.2532737Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:22:05.2533671Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:22:05.2534715Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:22:05.2535652Z tracer.run() 2024-08-20T22:22:05.2536404Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:22:05.2537138Z while self.step(): 2024-08-20T22:22:05.2537916Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:22:05.2538900Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:22:05.2539855Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:22:05.2540666Z return inner_fn(self, inst) 2024-08-20T22:22:05.2541587Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:22:05.2542424Z self.call_function(fn, args, {}) 2024-08-20T22:22:05.2543362Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:22:05.2544457Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:22:05.2545613Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:22:05.2546522Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:05.2547533Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:22:05.2548449Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:05.2549540Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:22:05.2550607Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:22:05.2551832Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:22:05.2552967Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:22:05.2554121Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:22:05.2555034Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:22:05.2556129Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:22:05.2556993Z tracer.run() 2024-08-20T22:22:05.2557936Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:22:05.2558718Z while self.step(): 2024-08-20T22:22:05.2559519Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:22:05.2560366Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:22:05.2561319Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:22:05.2562127Z return inner_fn(self, inst) 2024-08-20T22:22:05.2563157Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:22:05.2564380Z self.call_function(fn, args, kwargs) 2024-08-20T22:22:05.2565370Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:22:05.2566496Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:22:05.2567688Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:22:05.2568612Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:05.2569665Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:22:05.2570751Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:22:05.2572029Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:22:05.2573183Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:22:05.2574350Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:22:05.2575290Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:22:05.2576335Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:22:05.2577178Z tracer.run() 2024-08-20T22:22:05.2577971Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:22:05.2578769Z while self.step(): 2024-08-20T22:22:05.2579605Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:22:05.2580469Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:22:05.2581439Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:22:05.2582270Z return inner_fn(self, inst) 2024-08-20T22:22:05.2583156Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:22:05.2584085Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:22:05.2585385Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:22:05.2586469Z return handler(tx, args, kwargs) 2024-08-20T22:22:05.2587462Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:22:05.2588343Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:22:05.2589236Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:22:05.2590167Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:22:05.2591231Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:22:05.2592314Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:22:05.2593334Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:22:05.2594080Z raise UserError( # noqa: B904 2024-08-20T22:22:05.2596596Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:22:05.2598710Z 2024-08-20T22:22:05.2598959Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:22:05.2599915Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:05.2600658Z return node.target(*args, **kwargs) 2024-08-20T22:22:05.2600924Z 2024-08-20T22:22:05.2601112Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:22:05.2601837Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:22:05.2602733Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:05.2604149Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:22:05.2604972Z 2024-08-20T22:22:05.2605120Z User Stack (most recent call last): 2024-08-20T22:22:05.2605526Z (snipped, see stack below for prefix) 2024-08-20T22:22:05.2606519Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:22:05.2607400Z return self.inference(batched_inputs) 2024-08-20T22:22:05.2608400Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:22:05.2609328Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:22:05.2610413Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:22:05.2611346Z images = ImageList.from_tensors( 2024-08-20T22:22:05.2612342Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:22:05.2613427Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:22:05.2613838Z 2024-08-20T22:22:05.2614085Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:05.2615265Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:22:05.2616151Z 2024-08-20T22:22:05.2616248Z from user code: 2024-08-20T22:22:05.2617056Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:22:05.2617883Z return self.inference(batched_inputs) 2024-08-20T22:22:05.2618868Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:22:05.2619738Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:22:05.2620752Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:22:05.2621612Z images = ImageList.from_tensors( 2024-08-20T22:22:05.2622517Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:22:05.2623521Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:22:05.2623913Z 2024-08-20T22:22:05.2624181Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:22:05.2624652Z 2024-08-20T22:22:05.2624933Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:22:05.2625433Z fail_to_run 2024-08-20T22:22:05.2625998Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:22:05.2626650Z TIMING: 2024-08-20T22:22:05.2627171Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:40 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:22:05.2627973Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:22:08.3479315Z 2024-08-20T22:22:20.0398493Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:22:20.0399000Z loading model: 0it [00:11, ?it/s] 2024-08-20T22:22:20.0406472Z cpu eval detectron2_fasterrcnn_r_50_dc5 2024-08-20T22:22:34.4651226Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_50_dc5. Setting accuracy check to cosine 2024-08-20T22:22:34.5187163Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:22:49.3709585Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:04.1989468Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:12.3095147Z W0820 22:23:12.308994 16686 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:23:12.3097452Z E0820 22:23:12.309165 16686 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:23:12.3352671Z ERROR:common: 2024-08-20T22:23:12.3353023Z Traceback (most recent call last): 2024-08-20T22:23:12.3353993Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:12.3354802Z return node.target(*args, **kwargs) 2024-08-20T22:23:12.3361371Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:23:12.3362373Z r = self.shape_env.evaluate_expr( 2024-08-20T22:23:12.3363493Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:23:12.3364502Z return retlog(event.run(self)) 2024-08-20T22:23:12.3365433Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:23:12.3366284Z return self.f(*args, **kwargs) 2024-08-20T22:23:12.3367329Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:23:12.3368647Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:23:12.3370077Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:23:12.3371052Z raise self._make_data_dependent_error( 2024-08-20T22:23:12.3372622Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:23:12.3373653Z 2024-08-20T22:23:12.3373936Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:23:12.3374981Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:12.3375771Z return node.target(*args, **kwargs) 2024-08-20T22:23:12.3376072Z 2024-08-20T22:23:12.3376262Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:23:12.3377050Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:23:12.3378021Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:12.3379370Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:23:12.3380203Z 2024-08-20T22:23:12.3380337Z User Stack (most recent call last): 2024-08-20T22:23:12.3380760Z (snipped, see stack below for prefix) 2024-08-20T22:23:12.3381756Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:12.3382635Z return self.inference(batched_inputs) 2024-08-20T22:23:12.3383639Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:12.3384652Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:12.3385735Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:12.3386658Z images = ImageList.from_tensors( 2024-08-20T22:23:12.3387657Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:12.3388745Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:12.3389156Z 2024-08-20T22:23:12.3389400Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:12.3389812Z 2024-08-20T22:23:12.3390099Z The above exception was the direct cause of the following exception: 2024-08-20T22:23:12.3390555Z 2024-08-20T22:23:12.3390686Z Traceback (most recent call last): 2024-08-20T22:23:12.3391591Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:23:12.3392405Z ret_val = wrap_fake_exception( 2024-08-20T22:23:12.3393343Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:23:12.3394170Z return fn() 2024-08-20T22:23:12.3394942Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:23:12.3395829Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:23:12.3396803Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:23:12.3397690Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:23:12.3398673Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:12.3399465Z return node.target(*args, **kwargs) 2024-08-20T22:23:12.3400501Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:23:12.3401440Z r = self.shape_env.evaluate_expr( 2024-08-20T22:23:12.3402402Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:23:12.3403353Z return retlog(event.run(self)) 2024-08-20T22:23:12.3404268Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:23:12.3405186Z return self.f(*args, **kwargs) 2024-08-20T22:23:12.3406221Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:23:12.3407538Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:23:12.3408954Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:23:12.3409921Z raise self._make_data_dependent_error( 2024-08-20T22:23:12.3411196Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:23:12.3412828Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:23:12.3413458Z 2024-08-20T22:23:12.3413740Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:23:12.3414770Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:12.3415561Z return node.target(*args, **kwargs) 2024-08-20T22:23:12.3415859Z 2024-08-20T22:23:12.3416046Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:23:12.3416836Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:23:12.3417796Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:12.3418993Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:23:12.3419826Z 2024-08-20T22:23:12.3419956Z User Stack (most recent call last): 2024-08-20T22:23:12.3420378Z (snipped, see stack below for prefix) 2024-08-20T22:23:12.3421377Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:12.3422257Z return self.inference(batched_inputs) 2024-08-20T22:23:12.3423261Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:12.3424191Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:12.3425285Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:12.3426203Z images = ImageList.from_tensors( 2024-08-20T22:23:12.3427197Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:12.3428381Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:12.3428785Z 2024-08-20T22:23:12.3429038Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:12.3429432Z 2024-08-20T22:23:12.3429721Z During handling of the above exception, another exception occurred: 2024-08-20T22:23:12.3430154Z 2024-08-20T22:23:12.3430281Z Traceback (most recent call last): 2024-08-20T22:23:12.3431140Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:23:12.3431909Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:23:12.3432699Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:23:12.3433635Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:23:12.3434492Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:23:12.3435215Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:23:12.3436223Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:23:12.3437100Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:23:12.3438084Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:23:12.3438894Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:23:12.3439905Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:23:12.3440795Z return self._call_impl(*args, **kwargs) 2024-08-20T22:23:12.3441755Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:23:12.3442596Z return forward_call(*args, **kwargs) 2024-08-20T22:23:12.3443577Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:23:12.3444347Z return fn(*args, **kwargs) 2024-08-20T22:23:12.3445282Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:23:12.3446257Z return self._call_impl(*args, **kwargs) 2024-08-20T22:23:12.3447215Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:23:12.3448061Z return forward_call(*args, **kwargs) 2024-08-20T22:23:12.3448985Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:23:12.3449837Z return self._torchdynamo_orig_callable( 2024-08-20T22:23:12.3450790Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:23:12.3451593Z return _compile( 2024-08-20T22:23:12.3452411Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:23:12.3453369Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:23:12.3454470Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:23:12.3455520Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:23:12.3456514Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:23:12.3457351Z return function(*args, **kwargs) 2024-08-20T22:23:12.3458283Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:23:12.3459181Z out_code = transform_code_object(code, transform) 2024-08-20T22:23:12.3460290Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:23:12.3461277Z transformations(instructions, code_options) 2024-08-20T22:23:12.3462195Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:23:12.3462969Z return fn(*args, **kwargs) 2024-08-20T22:23:12.3463977Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:23:12.3464954Z tracer.run() 2024-08-20T22:23:12.3465772Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:23:12.3466570Z super().run() 2024-08-20T22:23:12.3467377Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:12.3468165Z while self.step(): 2024-08-20T22:23:12.3469004Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:12.3469863Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:12.3470845Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:12.3471668Z return inner_fn(self, inst) 2024-08-20T22:23:12.3472743Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:23:12.3473625Z self.call_function(fn, args, {}) 2024-08-20T22:23:12.3474595Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:23:12.3475715Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:23:12.3477009Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:23:12.3477927Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:12.3478949Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:23:12.3479844Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:12.3482225Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:23:12.3483561Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:23:12.3484843Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:23:12.3486003Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:23:12.3487172Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:23:12.3488104Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:23:12.3489147Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:23:12.3489990Z tracer.run() 2024-08-20T22:23:12.3490783Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:12.3491581Z while self.step(): 2024-08-20T22:23:12.3492417Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:12.3493282Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:12.3494256Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:12.3495094Z return inner_fn(self, inst) 2024-08-20T22:23:12.3496133Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:23:12.3496982Z self.call_function(fn, args, {}) 2024-08-20T22:23:12.3497919Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:23:12.3499000Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:23:12.3500167Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:23:12.3501076Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:12.3502095Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:23:12.3502986Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:12.3504004Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:23:12.3505065Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:23:12.3506303Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:23:12.3507494Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:23:12.3508597Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:23:12.3509533Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:23:12.3510503Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:23:12.3511286Z tracer.run() 2024-08-20T22:23:12.3512017Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:12.3512753Z while self.step(): 2024-08-20T22:23:12.3513529Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:12.3514326Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:12.3515220Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:12.3516052Z return inner_fn(self, inst) 2024-08-20T22:23:12.3516955Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:23:12.3517791Z self.call_function(fn, args, kwargs) 2024-08-20T22:23:12.3518683Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:23:12.3519723Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:23:12.3520821Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:23:12.3521686Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:12.3522642Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:23:12.3523928Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:23:12.3525215Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:23:12.3526375Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:23:12.3542047Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:23:12.3543085Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:23:12.3544192Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:23:12.3545014Z tracer.run() 2024-08-20T22:23:12.3545811Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:12.3546594Z while self.step(): 2024-08-20T22:23:12.3547409Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:12.3548252Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:12.3549218Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:12.3550030Z return inner_fn(self, inst) 2024-08-20T22:23:12.3550893Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:23:12.3551791Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:23:12.3552865Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:23:12.3553725Z return handler(tx, args, kwargs) 2024-08-20T22:23:12.3554766Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:23:12.3555763Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:23:12.3556669Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:23:12.3557920Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:23:12.3559046Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:23:12.3560100Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:23:12.3561160Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:23:12.3561959Z raise UserError( # noqa: B904 2024-08-20T22:23:12.3565123Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:23:12.3567565Z 2024-08-20T22:23:12.3567841Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:23:12.3568893Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:12.3569693Z return node.target(*args, **kwargs) 2024-08-20T22:23:12.3569980Z 2024-08-20T22:23:12.3570168Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:23:12.3570954Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:23:12.3571926Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:12.3573143Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:23:12.3573966Z 2024-08-20T22:23:12.3574099Z User Stack (most recent call last): 2024-08-20T22:23:12.3574527Z (snipped, see stack below for prefix) 2024-08-20T22:23:12.3575524Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:12.3576484Z return self.inference(batched_inputs) 2024-08-20T22:23:12.3577412Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:12.3578265Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:12.3579276Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:12.3580133Z images = ImageList.from_tensors( 2024-08-20T22:23:12.3581051Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:12.3582043Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:12.3582441Z 2024-08-20T22:23:12.3582670Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:12.3583769Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:23:12.3584521Z 2024-08-20T22:23:12.3584631Z from user code: 2024-08-20T22:23:12.3585425Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:12.3586248Z return self.inference(batched_inputs) 2024-08-20T22:23:12.3587177Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:12.3588039Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:12.3589047Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:12.3589890Z images = ImageList.from_tensors( 2024-08-20T22:23:12.3590815Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:12.3591936Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:12.3592321Z 2024-08-20T22:23:12.3592605Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:23:12.3593021Z 2024-08-20T22:23:12.3593290Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:23:12.3593805Z fail_to_run 2024-08-20T22:23:12.3594366Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:12.3595021Z TIMING: 2024-08-20T22:23:12.3595530Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:40 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:23:12.3596353Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:23:15.4583205Z 2024-08-20T22:23:22.3507651Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:23:22.3508338Z loading model: 0it [00:06, ?it/s] 2024-08-20T22:23:22.3518599Z cpu eval detectron2_fasterrcnn_r_50_fpn 2024-08-20T22:23:37.0539076Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_50_fpn. Setting accuracy check to cosine 2024-08-20T22:23:37.0884497Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:46.0270678Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:55.1068552Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:59.9078023Z W0820 22:23:59.906945 17013 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:23:59.9080329Z E0820 22:23:59.907114 17013 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:23:59.9333875Z ERROR:common: 2024-08-20T22:23:59.9334243Z Traceback (most recent call last): 2024-08-20T22:23:59.9335258Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:59.9336072Z return node.target(*args, **kwargs) 2024-08-20T22:23:59.9337716Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:23:59.9338773Z r = self.shape_env.evaluate_expr( 2024-08-20T22:23:59.9339776Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:23:59.9340641Z return retlog(event.run(self)) 2024-08-20T22:23:59.9341587Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:23:59.9342433Z return self.f(*args, **kwargs) 2024-08-20T22:23:59.9343481Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:23:59.9344796Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:23:59.9346241Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:23:59.9347214Z raise self._make_data_dependent_error( 2024-08-20T22:23:59.9348559Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:23:59.9349594Z 2024-08-20T22:23:59.9349936Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:23:59.9350979Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:59.9351795Z return node.target(*args, **kwargs) 2024-08-20T22:23:59.9352085Z 2024-08-20T22:23:59.9352289Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:23:59.9353276Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:23:59.9354260Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:59.9355481Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:23:59.9356312Z 2024-08-20T22:23:59.9356461Z User Stack (most recent call last): 2024-08-20T22:23:59.9356874Z (snipped, see stack below for prefix) 2024-08-20T22:23:59.9357880Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:59.9358772Z return self.inference(batched_inputs) 2024-08-20T22:23:59.9359780Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:59.9362240Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:59.9363471Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:59.9364588Z images = ImageList.from_tensors( 2024-08-20T22:23:59.9365599Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:59.9366693Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:59.9367111Z 2024-08-20T22:23:59.9367355Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:59.9367778Z 2024-08-20T22:23:59.9368067Z The above exception was the direct cause of the following exception: 2024-08-20T22:23:59.9368536Z 2024-08-20T22:23:59.9368669Z Traceback (most recent call last): 2024-08-20T22:23:59.9369583Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:23:59.9370403Z ret_val = wrap_fake_exception( 2024-08-20T22:23:59.9371353Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:23:59.9372178Z return fn() 2024-08-20T22:23:59.9372947Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:23:59.9373829Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:23:59.9374814Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:23:59.9375819Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:23:59.9376781Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:59.9377550Z return node.target(*args, **kwargs) 2024-08-20T22:23:59.9378573Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:23:59.9379497Z r = self.shape_env.evaluate_expr( 2024-08-20T22:23:59.9380438Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:23:59.9381279Z return retlog(event.run(self)) 2024-08-20T22:23:59.9382166Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:23:59.9382984Z return self.f(*args, **kwargs) 2024-08-20T22:23:59.9383988Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:23:59.9385266Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:23:59.9386647Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:23:59.9387590Z raise self._make_data_dependent_error( 2024-08-20T22:23:59.9388982Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:23:59.9390482Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:23:59.9391103Z 2024-08-20T22:23:59.9391381Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:23:59.9392378Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:59.9393157Z return node.target(*args, **kwargs) 2024-08-20T22:23:59.9393453Z 2024-08-20T22:23:59.9393640Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:23:59.9394416Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:23:59.9395446Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:59.9396739Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:23:59.9397515Z 2024-08-20T22:23:59.9397638Z User Stack (most recent call last): 2024-08-20T22:23:59.9398030Z (snipped, see stack below for prefix) 2024-08-20T22:23:59.9399145Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:59.9400005Z return self.inference(batched_inputs) 2024-08-20T22:23:59.9400990Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:59.9401900Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:59.9403224Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:59.9404151Z images = ImageList.from_tensors( 2024-08-20T22:23:59.9405161Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:59.9406248Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:59.9406666Z 2024-08-20T22:23:59.9406927Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:59.9407334Z 2024-08-20T22:23:59.9407632Z During handling of the above exception, another exception occurred: 2024-08-20T22:23:59.9408074Z 2024-08-20T22:23:59.9408207Z Traceback (most recent call last): 2024-08-20T22:23:59.9408926Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:23:59.9409690Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:23:59.9410485Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:23:59.9411429Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:23:59.9412292Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:23:59.9413012Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:23:59.9414030Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:23:59.9415015Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:23:59.9415877Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:23:59.9416635Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:23:59.9417573Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:23:59.9418397Z return self._call_impl(*args, **kwargs) 2024-08-20T22:23:59.9419283Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:23:59.9420072Z return forward_call(*args, **kwargs) 2024-08-20T22:23:59.9420963Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:23:59.9421671Z return fn(*args, **kwargs) 2024-08-20T22:23:59.9422557Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:23:59.9423404Z return self._call_impl(*args, **kwargs) 2024-08-20T22:23:59.9424290Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:23:59.9425074Z return forward_call(*args, **kwargs) 2024-08-20T22:23:59.9426125Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:23:59.9426958Z return self._torchdynamo_orig_callable( 2024-08-20T22:23:59.9427884Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:23:59.9428740Z return _compile( 2024-08-20T22:23:59.9429549Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:23:59.9430487Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:23:59.9431558Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:23:59.9432482Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:23:59.9433482Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:23:59.9434305Z return function(*args, **kwargs) 2024-08-20T22:23:59.9435243Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:23:59.9436143Z out_code = transform_code_object(code, transform) 2024-08-20T22:23:59.9437265Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:23:59.9438317Z transformations(instructions, code_options) 2024-08-20T22:23:59.9439187Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:23:59.9439930Z return fn(*args, **kwargs) 2024-08-20T22:23:59.9440922Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:23:59.9441712Z tracer.run() 2024-08-20T22:23:59.9442570Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:23:59.9443535Z super().run() 2024-08-20T22:23:59.9444347Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:59.9445138Z while self.step(): 2024-08-20T22:23:59.9445973Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:59.9446843Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:59.9447827Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:59.9448647Z return inner_fn(self, inst) 2024-08-20T22:23:59.9449599Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:23:59.9450481Z self.call_function(fn, args, {}) 2024-08-20T22:23:59.9451451Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:23:59.9452570Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:23:59.9453764Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:23:59.9454710Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:59.9455911Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:23:59.9456766Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:59.9457743Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:23:59.9458763Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:23:59.9459942Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:23:59.9461013Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:23:59.9462113Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:23:59.9463041Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:23:59.9464373Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:23:59.9465387Z tracer.run() 2024-08-20T22:23:59.9466191Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:59.9467037Z while self.step(): 2024-08-20T22:23:59.9467879Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:59.9468749Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:59.9469742Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:59.9470565Z return inner_fn(self, inst) 2024-08-20T22:23:59.9471516Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:23:59.9472397Z self.call_function(fn, args, {}) 2024-08-20T22:23:59.9473372Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:23:59.9474505Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:23:59.9475698Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:23:59.9476636Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:59.9477767Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:23:59.9478684Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:59.9479698Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:23:59.9480764Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:23:59.9481996Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:23:59.9483362Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:23:59.9484555Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:23:59.9485498Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:23:59.9486551Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:23:59.9487392Z tracer.run() 2024-08-20T22:23:59.9488200Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:59.9488996Z while self.step(): 2024-08-20T22:23:59.9489838Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:59.9490810Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:59.9491801Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:59.9492638Z return inner_fn(self, inst) 2024-08-20T22:23:59.9493611Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:23:59.9494507Z self.call_function(fn, args, kwargs) 2024-08-20T22:23:59.9495699Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:23:59.9496744Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:23:59.9497834Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:23:59.9498770Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:59.9499748Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:23:59.9500765Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:23:59.9501951Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:23:59.9503022Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:23:59.9504109Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:23:59.9504979Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:23:59.9505955Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:23:59.9506745Z tracer.run() 2024-08-20T22:23:59.9507485Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:59.9508228Z while self.step(): 2024-08-20T22:23:59.9509005Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:59.9509811Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:59.9510711Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:59.9511481Z return inner_fn(self, inst) 2024-08-20T22:23:59.9512302Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:23:59.9513167Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:23:59.9514182Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:23:59.9515009Z return handler(tx, args, kwargs) 2024-08-20T22:23:59.9516002Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:23:59.9516892Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:23:59.9517803Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:23:59.9518720Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:23:59.9519787Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:23:59.9520793Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:23:59.9521809Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:23:59.9522650Z raise UserError( # noqa: B904 2024-08-20T22:23:59.9525614Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:23:59.9527943Z 2024-08-20T22:23:59.9528214Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:23:59.9529256Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:59.9530058Z return node.target(*args, **kwargs) 2024-08-20T22:23:59.9530345Z 2024-08-20T22:23:59.9530549Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:23:59.9531332Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:23:59.9532366Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:59.9533600Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:23:59.9534429Z 2024-08-20T22:23:59.9534576Z User Stack (most recent call last): 2024-08-20T22:23:59.9535109Z (snipped, see stack below for prefix) 2024-08-20T22:23:59.9536018Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:59.9536843Z return self.inference(batched_inputs) 2024-08-20T22:23:59.9537775Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:59.9538635Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:59.9539631Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:59.9540492Z images = ImageList.from_tensors( 2024-08-20T22:23:59.9541420Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:59.9542427Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:59.9542807Z 2024-08-20T22:23:59.9543036Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:59.9544129Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:23:59.9544888Z 2024-08-20T22:23:59.9544985Z from user code: 2024-08-20T22:23:59.9545789Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:59.9546615Z return self.inference(batched_inputs) 2024-08-20T22:23:59.9547535Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:59.9548399Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:59.9549419Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:59.9550279Z images = ImageList.from_tensors( 2024-08-20T22:23:59.9551183Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:59.9552186Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:59.9552578Z 2024-08-20T22:23:59.9552848Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:23:59.9553262Z 2024-08-20T22:23:59.9553545Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:23:59.9554061Z fail_to_run 2024-08-20T22:23:59.9554605Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:59.9555261Z TIMING: 2024-08-20T22:23:59.9555783Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:43 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:23:59.9556674Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:24:03.0183567Z 2024-08-20T22:24:18.7476354Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:24:18.7476874Z loading model: 0it [00:15, ?it/s] 2024-08-20T22:24:18.7477279Z Traceback (most recent call last): 2024-08-20T22:24:18.7477923Z File "benchmarks/dynamo/torchbench.py", line 460, in 2024-08-20T22:24:18.7478513Z torchbench_main() 2024-08-20T22:24:18.7479017Z File "benchmarks/dynamo/torchbench.py", line 456, in torchbench_main 2024-08-20T22:24:18.7479650Z main(TorchBenchmarkRunner(), original_dir) 2024-08-20T22:24:18.7480372Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4087, in main 2024-08-20T22:24:18.7483384Z process_entry(0, runner, original_dir, args) 2024-08-20T22:24:18.7484476Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4019, in process_entry 2024-08-20T22:24:18.7488022Z return run(runner, args, original_dir) 2024-08-20T22:24:18.7488774Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4661, in run 2024-08-20T22:24:18.7493593Z assert marked, f"nothing in example_inputs had a dim with {batch_size}" 2024-08-20T22:24:18.7494353Z AssertionError: nothing in example_inputs had a dim with 4 2024-08-20T22:24:19.4027253Z Run failed with return code: 1 2024-08-20T22:24:19.4027727Z Output: None 2024-08-20T22:24:19.4028012Z Error: None 2024-08-20T22:24:21.9001565Z 2024-08-20T22:24:30.7731630Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:24:30.7732111Z loading model: 0it [00:08, ?it/s] 2024-08-20T22:24:30.7732583Z Traceback (most recent call last): 2024-08-20T22:24:30.7733123Z File "benchmarks/dynamo/torchbench.py", line 460, in 2024-08-20T22:24:30.7735082Z torchbench_main() 2024-08-20T22:24:30.7735589Z File "benchmarks/dynamo/torchbench.py", line 456, in torchbench_main 2024-08-20T22:24:30.7736237Z main(TorchBenchmarkRunner(), original_dir) 2024-08-20T22:24:30.7736971Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4087, in main 2024-08-20T22:24:30.7741008Z process_entry(0, runner, original_dir, args) 2024-08-20T22:24:30.7741766Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4019, in process_entry 2024-08-20T22:24:30.7745700Z return run(runner, args, original_dir) 2024-08-20T22:24:30.7746396Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4661, in run 2024-08-20T22:24:30.7751297Z assert marked, f"nothing in example_inputs had a dim with {batch_size}" 2024-08-20T22:24:30.7751989Z AssertionError: nothing in example_inputs had a dim with 4 2024-08-20T22:24:31.4059890Z Run failed with return code: 1 2024-08-20T22:24:31.4060369Z Output: None 2024-08-20T22:24:31.4060660Z Error: None 2024-08-20T22:24:33.9006158Z 2024-08-20T22:24:48.7465640Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:24:48.7466132Z loading model: 0it [00:14, ?it/s] 2024-08-20T22:24:48.7466523Z Traceback (most recent call last): 2024-08-20T22:24:48.7468156Z File "benchmarks/dynamo/torchbench.py", line 460, in 2024-08-20T22:24:48.7468739Z torchbench_main() 2024-08-20T22:24:48.7469249Z File "benchmarks/dynamo/torchbench.py", line 456, in torchbench_main 2024-08-20T22:24:48.7469885Z main(TorchBenchmarkRunner(), original_dir) 2024-08-20T22:24:48.7470599Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4087, in main 2024-08-20T22:24:48.7471701Z process_entry(0, runner, original_dir, args) 2024-08-20T22:24:48.7472526Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4019, in process_entry 2024-08-20T22:24:48.7475901Z return run(runner, args, original_dir) 2024-08-20T22:24:48.7476581Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4661, in run 2024-08-20T22:24:48.7481348Z assert marked, f"nothing in example_inputs had a dim with {batch_size}" 2024-08-20T22:24:48.7482060Z AssertionError: nothing in example_inputs had a dim with 4 2024-08-20T22:24:49.3247986Z Run failed with return code: 1 2024-08-20T22:24:49.3248546Z Output: None 2024-08-20T22:24:49.3248824Z Error: None 2024-08-20T22:24:51.8120333Z 2024-08-20T22:24:59.0441165Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:24:59.0441681Z loading model: 0it [00:07, ?it/s] 2024-08-20T22:24:59.0442093Z Traceback (most recent call last): 2024-08-20T22:24:59.0442632Z File "benchmarks/dynamo/torchbench.py", line 460, in 2024-08-20T22:24:59.0443268Z torchbench_main() 2024-08-20T22:24:59.0443772Z File "benchmarks/dynamo/torchbench.py", line 456, in torchbench_main 2024-08-20T22:24:59.0444406Z main(TorchBenchmarkRunner(), original_dir) 2024-08-20T22:24:59.0445131Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4087, in main 2024-08-20T22:24:59.0447088Z process_entry(0, runner, original_dir, args) 2024-08-20T22:24:59.0448169Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4019, in process_entry 2024-08-20T22:24:59.0451518Z return run(runner, args, original_dir) 2024-08-20T22:24:59.0452209Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4661, in run 2024-08-20T22:24:59.0456914Z assert marked, f"nothing in example_inputs had a dim with {batch_size}" 2024-08-20T22:24:59.0457624Z AssertionError: nothing in example_inputs had a dim with 4 2024-08-20T22:24:59.6242665Z Run failed with return code: 1 2024-08-20T22:24:59.6243359Z Output: None 2024-08-20T22:24:59.6243639Z Error: None 2024-08-20T22:25:02.2333774Z 2024-08-20T22:25:10.5758383Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:25:10.5759154Z loading model: 0it [00:08, ?it/s] 2024-08-20T22:25:10.5759811Z cpu eval dlrm 2024-08-20T22:25:11.2289696Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:25:11.4528177Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:25:11.6746962Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:25:37.0718067Z Error: weights_offset must be aligned to 16K boundary 2024-08-20T22:25:37.0721587Z ERROR:common: 2024-08-20T22:25:37.0722119Z Traceback (most recent call last): 2024-08-20T22:25:37.0723329Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:25:37.0728037Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:25:37.0728906Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:25:37.0729871Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:25:37.0730736Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1380, in load 2024-08-20T22:25:37.0731498Z cls.cache[key] = torch._export.aot_load(so_path, device) 2024-08-20T22:25:37.0732712Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_export/__init__.py", line 300, in aot_load 2024-08-20T22:25:37.0733892Z runner = torch._C._aoti.AOTIModelContainerRunnerCpu(so_path, 1) # type: ignore[call-arg] 2024-08-20T22:25:37.0735666Z RuntimeError: create_func_( &container_handle_, num_models, device_str.c_str(), cubin_dir.empty() ? nullptr : cubin_dir.c_str()) API call failed at /var/lib/jenkins/workspace/torch/csrc/inductor/aoti_runner/model_container_runner.cpp, line 70 2024-08-20T22:25:37.0737333Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:25:37.0737879Z fail_to_run 2024-08-20T22:25:37.0738480Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:25:37.0739419Z TIMING: backend_compile:0.0001 entire_frame_compile:0.259 total_wall_time:0.259 2024-08-20T22:25:37.0740595Z STATS: call_* op count: 36 | FakeTensor.__torch_dispatch__:187 | FakeTensorMode.__torch_dispatch__:2410 | ProxyTorchDispatchMode.__torch_dispatch__:546 2024-08-20T22:25:37.0741943Z Dynamo produced 1 graphs covering 36 ops with 0 graph breaks (0 unique) 2024-08-20T22:25:40.5010153Z 2024-08-20T22:25:41.7336430Z loading model: 0it [00:00, ?it/s]Downloading https://doctr-static.mindee.com/models?id=v0.7.0/db_resnet50-79bd7d70.pt&src=0 to /var/lib/jenkins/.cache/doctr/models/db_resnet50-79bd7d70.pt 2024-08-20T22:25:42.0753802Z 2024-08-20T22:25:42.0753887Z 2024-08-20T22:25:42.1754999Z 0% 0/102021912 [00:00 2024-08-20T22:26:17.4972409Z [ 2024-08-20T22:26:17.4973202Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/detection/core.py", line 92, in 2024-08-20T22:26:17.4974213Z cv2.morphologyEx(bmap[..., idx], cv2.MORPH_OPEN, self._opening_kernel) 2024-08-20T22:26:17.4974667Z 2024-08-20T22:26:17.4974968Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:26:17.4975420Z 2024-08-20T22:26:17.4975724Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:26:17.4976282Z fail_to_run 2024-08-20T22:26:17.4976886Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:17.4977588Z TIMING: 2024-08-20T22:26:17.4978163Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:605 | FakeTensorMode.__torch_dispatch__:3780 2024-08-20T22:26:17.4979128Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:26:20.6681891Z 2024-08-20T22:26:24.1101256Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:24.1101843Z loading model: 0it [00:03, ?it/s] 2024-08-20T22:26:24.1108267Z cpu eval doctr_reco_predictor 2024-08-20T22:26:24.3485866Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:24.3933822Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:24.4331421Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:37.1826578Z W0820 22:26:37.181843 19247 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Ne(u0, 123), hint=None, expect_rational=True, size_oblivious=False, forcing_spec=False 2024-08-20T22:26:37.1828794Z E0820 22:26:37.181993 19247 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Ne(u0, 123), None), **{'fx_node': ne_217}) 2024-08-20T22:26:37.1885484Z ERROR:common: 2024-08-20T22:26:37.1885836Z Traceback (most recent call last): 2024-08-20T22:26:37.1886903Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/tensor.py", line 1103, in evaluate_expr 2024-08-20T22:26:37.1887805Z return guard_scalar(self.sym_num) 2024-08-20T22:26:37.1888869Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 845, in guard_scalar 2024-08-20T22:26:37.1889793Z return guard_bool(a) 2024-08-20T22:26:37.1890750Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1018, in guard_bool 2024-08-20T22:26:37.1891792Z return a.node.guard_bool("", 0) # NB: uses Python backtrace 2024-08-20T22:26:37.1892898Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 451, in guard_bool 2024-08-20T22:26:37.1893984Z r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node) 2024-08-20T22:26:37.1895244Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:26:37.1896108Z return retlog(event.run(self)) 2024-08-20T22:26:37.1897029Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:26:37.1897854Z return self.f(*args, **kwargs) 2024-08-20T22:26:37.1898886Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:26:37.1900194Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:26:37.1901628Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5168, in _evaluate_expr 2024-08-20T22:26:37.1902595Z concrete_val = compute_concrete_val() 2024-08-20T22:26:37.1903892Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5149, in compute_concrete_val 2024-08-20T22:26:37.1904886Z return self.size_hint(orig_expr) 2024-08-20T22:26:37.1905917Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 4671, in size_hint 2024-08-20T22:26:37.1906934Z raise self._make_data_dependent_error(result_expr, expr) 2024-08-20T22:26:37.1908351Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Ne(u0, 123) (unhinted: Ne(u0, 123)). (Size-like symbols: none) 2024-08-20T22:26:37.1909400Z 2024-08-20T22:26:37.1909669Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:26:37.1910807Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/tensor.py", line 1103, in evaluate_expr 2024-08-20T22:26:37.1911817Z return guard_scalar(self.sym_num) 2024-08-20T22:26:37.1912095Z 2024-08-20T22:26:37.1912303Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:26:37.1913082Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:26:37.1914051Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:37.1915274Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:26:37.1916098Z 2024-08-20T22:26:37.1916243Z User Stack (most recent call last): 2024-08-20T22:26:37.1916650Z (snipped, see stack below for prefix) 2024-08-20T22:26:37.1917678Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 224, in forward 2024-08-20T22:26:37.1918617Z out["preds"] = self.postprocessor(logits) 2024-08-20T22:26:37.1919671Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 99, in __call__ 2024-08-20T22:26:37.1920778Z return self.ctc_best_path(logits=logits, vocab=self.vocab, blank=len(self.vocab)) 2024-08-20T22:26:37.1922144Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 78, in ctc_best_path 2024-08-20T22:26:37.1923053Z words = [ 2024-08-20T22:26:37.1923968Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-08-20T22:26:37.1925070Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-08-20T22:26:37.1926277Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-08-20T22:26:37.1927368Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-08-20T22:26:37.1927851Z 2024-08-20T22:26:37.1928097Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:37.1928494Z 2024-08-20T22:26:37.1928792Z During handling of the above exception, another exception occurred: 2024-08-20T22:26:37.1929234Z 2024-08-20T22:26:37.1929366Z Traceback (most recent call last): 2024-08-20T22:26:37.1930292Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:26:37.1931108Z tracer.run() 2024-08-20T22:26:37.1931918Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:26:37.1932699Z super().run() 2024-08-20T22:26:37.1933500Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:26:37.1934292Z while self.step(): 2024-08-20T22:26:37.1935126Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:26:37.1935995Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:26:37.1937031Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:26:37.1937863Z return inner_fn(self, inst) 2024-08-20T22:26:37.1938817Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:26:37.1939698Z self.call_function(fn, args, {}) 2024-08-20T22:26:37.1940650Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:26:37.1941772Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:26:37.1942977Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/user_defined.py", line 876, in call_function 2024-08-20T22:26:37.1943952Z return self.call_method(tx, "__call__", args, kwargs) 2024-08-20T22:26:37.1945085Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/user_defined.py", line 736, in call_method 2024-08-20T22:26:37.1946156Z return UserMethodVariable(method, self, source=source).call_function( 2024-08-20T22:26:37.1947332Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:26:37.1948263Z return super().call_function(tx, args, kwargs) 2024-08-20T22:26:37.1949307Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:26:37.1950223Z return super().call_function(tx, args, kwargs) 2024-08-20T22:26:37.1951274Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:26:37.1952367Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:26:37.1953654Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:26:37.1954799Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:26:37.1955987Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:26:37.1956925Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:26:37.1957979Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:26:37.1958814Z tracer.run() 2024-08-20T22:26:37.1959621Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:26:37.1960426Z while self.step(): 2024-08-20T22:26:37.1961274Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:26:37.1962224Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:26:37.1963203Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:26:37.1964220Z return inner_fn(self, inst) 2024-08-20T22:26:37.1965207Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:26:37.1966119Z self.call_function(fn, args, kwargs) 2024-08-20T22:26:37.1967100Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:26:37.1968235Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:26:37.1969432Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:26:37.1970368Z return super().call_function(tx, args, kwargs) 2024-08-20T22:26:37.1971407Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:26:37.1972645Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:26:37.1973931Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:26:37.1975096Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:26:37.1976384Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:26:37.1977282Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:26:37.1978381Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:26:37.1979166Z tracer.run() 2024-08-20T22:26:37.1979911Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:26:37.1980713Z while self.step(): 2024-08-20T22:26:37.1981495Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:26:37.1982298Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:26:37.1983204Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:26:37.1983955Z return inner_fn(self, inst) 2024-08-20T22:26:37.1984832Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:26:37.1985641Z self.call_function(fn, args, {}) 2024-08-20T22:26:37.1986535Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:26:37.1987758Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:26:37.1988922Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:26:37.1989983Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:26:37.1991220Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:26:37.1992342Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:26:37.1993479Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:26:37.1994383Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:26:37.1995404Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:26:37.1996225Z tracer.run() 2024-08-20T22:26:37.1997001Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:26:37.1997783Z while self.step(): 2024-08-20T22:26:37.1998604Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:26:37.1999527Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:26:37.2000420Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:26:37.2001189Z return inner_fn(self, inst) 2024-08-20T22:26:37.2002132Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:26:37.2003157Z self.call_function(fn, args, {}) 2024-08-20T22:26:37.2004112Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:26:37.2005231Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:26:37.2006433Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:26:37.2007620Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:26:37.2008899Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:26:37.2010041Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:26:37.2011228Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:26:37.2012162Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:26:37.2013208Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:26:37.2014037Z tracer.run() 2024-08-20T22:26:37.2014932Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:26:37.2015719Z while self.step(): 2024-08-20T22:26:37.2016493Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:26:37.2017280Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:26:37.2018175Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 532, in inner 2024-08-20T22:26:37.2018987Z eval_result = value.evaluate_expr(self.output) 2024-08-20T22:26:37.2019949Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/tensor.py", line 1105, in evaluate_expr 2024-08-20T22:26:37.2020758Z raise UserError( # noqa: B904 2024-08-20T22:26:37.2021993Z torch._dynamo.exc.UserError: Consider annotating your code using torch._check*(). Could not guard on data-dependent expression Ne(u0, 123) (unhinted: Ne(u0, 123)). (Size-like symbols: none) 2024-08-20T22:26:37.2022993Z 2024-08-20T22:26:37.2023411Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:26:37.2024520Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/tensor.py", line 1103, in evaluate_expr 2024-08-20T22:26:37.2025375Z return guard_scalar(self.sym_num) 2024-08-20T22:26:37.2025647Z 2024-08-20T22:26:37.2025830Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:26:37.2026601Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:26:37.2027547Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:37.2028728Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:26:37.2029525Z 2024-08-20T22:26:37.2029668Z User Stack (most recent call last): 2024-08-20T22:26:37.2030064Z (snipped, see stack below for prefix) 2024-08-20T22:26:37.2031064Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 224, in forward 2024-08-20T22:26:37.2031982Z out["preds"] = self.postprocessor(logits) 2024-08-20T22:26:37.2032996Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 99, in __call__ 2024-08-20T22:26:37.2049207Z return self.ctc_best_path(logits=logits, vocab=self.vocab, blank=len(self.vocab)) 2024-08-20T22:26:37.2050557Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 78, in ctc_best_path 2024-08-20T22:26:37.2051479Z words = [ 2024-08-20T22:26:37.2052397Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-08-20T22:26:37.2053479Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-08-20T22:26:37.2054698Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-08-20T22:26:37.2055810Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-08-20T22:26:37.2056398Z 2024-08-20T22:26:37.2056663Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:37.2057840Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:26:37.2058670Z 2024-08-20T22:26:37.2058957Z During handling of the above exception, another exception occurred: 2024-08-20T22:26:37.2059415Z 2024-08-20T22:26:37.2059549Z Traceback (most recent call last): 2024-08-20T22:26:37.2060508Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:26:37.2061403Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:26:37.2062321Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1483, in inner 2024-08-20T22:26:37.2063185Z raise constraint_violation_error 2024-08-20T22:26:37.2064265Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:26:37.2065078Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:26:37.2066079Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:26:37.2066983Z return self._call_impl(*args, **kwargs) 2024-08-20T22:26:37.2067942Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:26:37.2068780Z return forward_call(*args, **kwargs) 2024-08-20T22:26:37.2069642Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:26:37.2070415Z return fn(*args, **kwargs) 2024-08-20T22:26:37.2071366Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:26:37.2072279Z return self._call_impl(*args, **kwargs) 2024-08-20T22:26:37.2073218Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:26:37.2074059Z return forward_call(*args, **kwargs) 2024-08-20T22:26:37.2075000Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:26:37.2075851Z return self._torchdynamo_orig_callable( 2024-08-20T22:26:37.2076893Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:26:37.2077668Z return _compile( 2024-08-20T22:26:37.2078478Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:26:37.2079498Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:26:37.2080511Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:26:37.2081382Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:26:37.2082567Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:26:37.2083410Z return function(*args, **kwargs) 2024-08-20T22:26:37.2084369Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:26:37.2085275Z out_code = transform_code_object(code, transform) 2024-08-20T22:26:37.2086423Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:26:37.2087434Z transformations(instructions, code_options) 2024-08-20T22:26:37.2088373Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:26:37.2089154Z return fn(*args, **kwargs) 2024-08-20T22:26:37.2090158Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 630, in transform 2024-08-20T22:26:37.2091003Z bisect(tracer.output.shape_env) 2024-08-20T22:26:37.2091971Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/validator.py", line 723, in bisect 2024-08-20T22:26:37.2093058Z last_exception = check_shapeenv_fails(shape_env, shape_env._snapshot_tracked_fakes()) 2024-08-20T22:26:37.2094349Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/validator.py", line 705, in check_shapeenv_fails 2024-08-20T22:26:37.2095379Z shape_env.produce_guards( 2024-08-20T22:26:37.2096370Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 4185, in produce_guards 2024-08-20T22:26:37.2097309Z raise ConstraintViolationError( 2024-08-20T22:26:37.2098559Z torch.fx.experimental.symbolic_shapes.ConstraintViolationError: Constraints violated (L['x'].size()[0])! For more information, run with TORCH_LOGS="+dynamic". 2024-08-20T22:26:37.2100136Z - Not all values of RelaxedUnspecConstraint(L['x'].size()[0]) are valid because L['x'].size()[0] was inferred to be a constant (4). 2024-08-20T22:26:37.2100835Z 2024-08-20T22:26:37.2100839Z 2024-08-20T22:26:37.2101104Z During handling of the above exception, another exception occurred: 2024-08-20T22:26:37.2101516Z 2024-08-20T22:26:37.2101655Z Traceback (most recent call last): 2024-08-20T22:26:37.2102327Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:26:37.2103023Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:26:37.2103758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:26:37.2104639Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:26:37.2105426Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:26:37.2106089Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:26:37.2107022Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 569, in _export_to_torch_ir 2024-08-20T22:26:37.2107998Z raise UserError(UserErrorType.CONSTRAINT_VIOLATION, str(e)) # noqa: B904 2024-08-20T22:26:37.2109066Z torch._dynamo.exc.UserError: Constraints violated (L['x'].size()[0])! For more information, run with TORCH_LOGS="+dynamic". 2024-08-20T22:26:37.2110370Z - Not all values of RelaxedUnspecConstraint(L['x'].size()[0]) are valid because L['x'].size()[0] was inferred to be a constant (4). 2024-08-20T22:26:37.2111057Z 2024-08-20T22:26:37.2111323Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:26:37.2111837Z fail_to_run 2024-08-20T22:26:37.2112398Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:37.2113059Z TIMING: 2024-08-20T22:26:37.2113790Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:260 | FakeTensorMode.__torch_dispatch__:12549 | attempt fast:957 | fast is_contiguous:957 2024-08-20T22:26:37.2114812Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:26:40.6441561Z 2024-08-20T22:26:41.3998460Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:41.3999038Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:41.3999591Z cpu eval drq 2024-08-20T22:26:41.4084794Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:41.4146839Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:41.4204970Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:41.7897349Z ERROR:common: 2024-08-20T22:26:41.7897715Z Traceback (most recent call last): 2024-08-20T22:26:41.7898489Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:26:41.7899455Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:26:41.7900271Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:26:41.7901243Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:26:41.7902093Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:26:41.7902817Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:26:41.7903998Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:26:41.7904883Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:26:41.7905813Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1574, in inner 2024-08-20T22:26:41.7906606Z graph = rewrite_signature( 2024-08-20T22:26:41.7907666Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1077, in rewrite_signature 2024-08-20T22:26:41.7908723Z check_user_input_output(flat_results_traced, UserErrorType.INVALID_OUTPUT) 2024-08-20T22:26:41.7909933Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1063, in check_user_input_output 2024-08-20T22:26:41.7910796Z raise UserError( 2024-08-20T22:26:41.7912053Z torch._dynamo.exc.UserError: It looks like one of the outputs with type `` is not supported or pytree-flattenable. 2024-08-20T22:26:41.7915757Z Exported graphs outputs can only contain the following supported types: [, , , , , , , , , , , , , , , , , , ]. 2024-08-20T22:26:41.7919207Z If you are using a custom class object, please register a pytree_flatten/unflatten function using `torch.utils._pytree.register_pytree_node` or `torch.export.register_dataclass`. 2024-08-20T22:26:41.7920498Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:26:41.7921043Z fail_to_run 2024-08-20T22:26:41.7921637Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:41.7922912Z TIMING: backend_compile:8e-05 entire_frame_compile:0.21563 total_wall_time:0.21563 2024-08-20T22:26:41.7923852Z STATS: call_* op count: 29 | FakeTensor.__torch_dispatch__:66 | FakeTensorMode.__torch_dispatch__:219 2024-08-20T22:26:41.7924751Z Dynamo produced 1 graphs covering 29 ops with 0 graph breaks (0 unique) 2024-08-20T22:26:44.7793236Z 2024-08-20T22:26:45.1635679Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:45.1636291Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:45.1640882Z cpu eval functorch_dp_cifar10 2024-08-20T22:26:45.2049519Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:45.2272189Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:45.2487330Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:26.2612411Z pass 2024-08-20T22:27:26.2614933Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:26.2617311Z TIMING: backend_compile:9e-05 entire_frame_compile:1.4586 total_wall_time:1.4586 2024-08-20T22:27:26.2619610Z STATS: call_* op count: 69 | FakeTensor.__torch_dispatch__:1294 | FakeTensorMode.__torch_dispatch__:13525 | attempt fast:1023 | fast is_contiguous:731 | ProxyTorchDispatchMode.__torch_dispatch__:1200 | slow no contiguity match:196 | fast channels_last:96 2024-08-20T22:27:26.2622061Z Dynamo produced 1 graphs covering 69 ops with 0 graph breaks (0 unique) 2024-08-20T22:27:29.8725433Z 2024-08-20T22:27:30.0202016Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:27:30.0202619Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:27:30.0203266Z cpu eval functorch_maml_omniglot 2024-08-20T22:27:30.0247968Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:30.0293474Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:30.0328092Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:51.1031369Z /tmp/tmp14ep68yx/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/c3uudlvuyy6lfrc77kwamuwlfvtd7ywzgrwtndbvxnho5rhqzzn3.cpp: In function ‘void cpp_fused__native_batch_norm_legit_max_pool2d_with_indices_relu_0(const float*, const float*, const float*, float*, float*, float*, float*)’: 2024-08-20T22:27:51.1034907Z /tmp/tmp14ep68yx/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/c3uudlvuyy6lfrc77kwamuwlfvtd7ywzgrwtndbvxnho5rhqzzn3.cpp:480:36: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:27:51.1036478Z 480 | Welford tmp_acc0 = Welford(); 2024-08-20T22:27:51.1037009Z | ^~~~~~~~ 2024-08-20T22:27:51.1039099Z /tmp/tmp14ep68yx/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/c3uudlvuyy6lfrc77kwamuwlfvtd7ywzgrwtndbvxnho5rhqzzn3.cpp: In function ‘void cpp_fused__native_batch_norm_legit_max_pool2d_with_indices_relu_1(const float*, const float*, const float*, float*, float*, float*, float*)’: 2024-08-20T22:27:51.1042157Z /tmp/tmp14ep68yx/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/c3uudlvuyy6lfrc77kwamuwlfvtd7ywzgrwtndbvxnho5rhqzzn3.cpp:562:32: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:27:51.1043823Z 562 | Welford tmp_acc0 = Welford(); 2024-08-20T22:27:51.1044336Z | ^~~~~~~~ 2024-08-20T22:27:51.1046459Z /tmp/tmp14ep68yx/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/c3uudlvuyy6lfrc77kwamuwlfvtd7ywzgrwtndbvxnho5rhqzzn3.cpp: In function ‘void cpp_fused__native_batch_norm_legit_max_pool2d_with_indices_relu_2(const float*, const float*, const float*, float*, float*, float*, float*, float*)’: 2024-08-20T22:27:51.1049552Z /tmp/tmp14ep68yx/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/c3uudlvuyy6lfrc77kwamuwlfvtd7ywzgrwtndbvxnho5rhqzzn3.cpp:645:32: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:27:51.1051103Z 645 | Welford tmp_acc0 = Welford(); 2024-08-20T22:27:51.1051616Z | ^~~~~~~~ 2024-08-20T22:27:51.3650504Z pass 2024-08-20T22:27:51.3653333Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:51.3654958Z TIMING: backend_compile:9e-05 entire_frame_compile:0.16005 total_wall_time:0.16005 2024-08-20T22:27:51.3656560Z STATS: call_* op count: 14 | FakeTensor.__torch_dispatch__:208 | FakeTensorMode.__torch_dispatch__:1958 | ProxyTorchDispatchMode.__torch_dispatch__:331 2024-08-20T22:27:51.3658082Z Dynamo produced 1 graphs covering 14 ops with 0 graph breaks (0 unique) 2024-08-20T22:27:54.6071803Z 2024-08-20T22:27:56.0085648Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:27:56.0086143Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:27:56.0086591Z cpu eval hf_Albert 2024-08-20T22:27:57.2900756Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:57.9272745Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:58.5732270Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:04.6770454Z pass 2024-08-20T22:29:04.6774921Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:04.6776266Z TIMING: backend_compile:0.00017 entire_frame_compile:5.34076 total_wall_time:5.34076 2024-08-20T22:29:04.6778165Z STATS: call_* op count: 438 | FakeTensorMode.__torch_dispatch__:37059 | attempt fast:3571 | fast is_contiguous:3571 | FakeTensor.__torch_dispatch__:3511 | ProxyTorchDispatchMode.__torch_dispatch__:5819 2024-08-20T22:29:04.6780021Z Dynamo produced 1 graphs covering 438 ops with 0 graph breaks (0 unique) 2024-08-20T22:29:09.0518595Z 2024-08-20T22:29:13.3318839Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:13.3319313Z loading model: 0it [00:04, ?it/s] 2024-08-20T22:29:13.3333111Z cpu eval hf_Bart 2024-08-20T22:29:14.9193854Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:15.5649898Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:16.2847421Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:50.3838501Z pass 2024-08-20T22:30:50.3840400Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:50.3841388Z TIMING: backend_compile:0.00016 entire_frame_compile:5.34496 total_wall_time:5.34496 2024-08-20T22:30:50.3843063Z STATS: call_* op count: 470 | FakeTensorMode.__torch_dispatch__:42492 | FakeTensor.__torch_dispatch__:4246 | attempt fast:3545 | fast is_contiguous:3544 | slow no contiguity match:1 | ProxyTorchDispatchMode.__torch_dispatch__:6688 2024-08-20T22:30:50.3844598Z Dynamo produced 1 graphs covering 470 ops with 0 graph breaks (0 unique) 2024-08-20T22:30:54.7942577Z 2024-08-20T22:30:57.4652170Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:57.4652680Z loading model: 0it [00:02, ?it/s] 2024-08-20T22:30:57.4669020Z cpu eval hf_Bert 2024-08-20T22:30:59.2310847Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:59.8417274Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:31:00.4564213Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:32:23.9436625Z pass 2024-08-20T22:32:23.9438674Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:32:23.9439717Z TIMING: backend_compile:0.00015 entire_frame_compile:6.76083 total_wall_time:6.76083 2024-08-20T22:32:23.9441173Z STATS: call_* op count: 368 | FakeTensorMode.__torch_dispatch__:38633 | attempt fast:3366 | fast is_contiguous:3366 | FakeTensor.__torch_dispatch__:3404 | ProxyTorchDispatchMode.__torch_dispatch__:6318 2024-08-20T22:32:23.9442709Z Dynamo produced 1 graphs covering 368 ops with 0 graph breaks (0 unique) 2024-08-20T22:32:28.3445862Z 2024-08-20T22:32:33.8365563Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:32:33.8366034Z loading model: 0it [00:05, ?it/s] 2024-08-20T22:32:33.8395702Z cpu eval hf_Bert_large 2024-08-20T22:32:39.1773415Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:32:40.8249691Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:32:42.4451489Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:35:22.7122555Z pass 2024-08-20T22:35:22.7125203Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:35:22.7126216Z TIMING: backend_compile:0.00025 entire_frame_compile:13.05402 total_wall_time:13.05402 2024-08-20T22:35:22.7127967Z STATS: call_* op count: 716 | FakeTensorMode.__torch_dispatch__:75360 | attempt fast:6541 | fast is_contiguous:6541 | FakeTensor.__torch_dispatch__:6632 | ProxyTorchDispatchMode.__torch_dispatch__:12330 2024-08-20T22:35:22.7129369Z Dynamo produced 1 graphs covering 716 ops with 0 graph breaks (0 unique) 2024-08-20T22:35:28.2280347Z 2024-08-20T22:35:30.1710880Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:35:30.1711344Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:35:30.1715356Z cpu eval hf_DistilBert 2024-08-20T22:35:31.2370152Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:35:31.6506805Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:35:32.0496778Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:36:29.6682612Z pass 2024-08-20T22:36:29.6685464Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:36:29.6687211Z TIMING: backend_compile:0.00012 entire_frame_compile:2.87382 total_wall_time:2.87382 2024-08-20T22:36:29.6690205Z STATS: call_* op count: 205 | FakeTensor.__torch_dispatch__:2033 | FakeTensorMode.__torch_dispatch__:21111 | attempt fast:1664 | fast is_contiguous:1616 | slow no contiguity match:48 | ProxyTorchDispatchMode.__torch_dispatch__:3255 2024-08-20T22:36:29.6693128Z Dynamo produced 1 graphs covering 205 ops with 0 graph breaks (0 unique) 2024-08-20T22:36:33.5623875Z 2024-08-20T22:36:36.4358407Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:36:36.4358903Z loading model: 0it [00:02, ?it/s] 2024-08-20T22:36:36.4368094Z cpu eval hf_GPT2 2024-08-20T22:36:37.1354448Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:36:37.3529350Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:36:37.5677424Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:38:02.0727904Z pass 2024-08-20T22:38:02.0728636Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:38:02.0729601Z TIMING: backend_compile:0.00024 entire_frame_compile:5.83979 total_wall_time:5.83979 2024-08-20T22:38:02.0731045Z STATS: call_* op count: 623 | FakeTensorMode.__torch_dispatch__:33711 | FakeTensor.__torch_dispatch__:2436 | attempt fast:2906 | fast is_contiguous:2906 | ProxyTorchDispatchMode.__torch_dispatch__:5631 2024-08-20T22:38:02.0732427Z Dynamo produced 1 graphs covering 623 ops with 0 graph breaks (0 unique) 2024-08-20T22:38:06.3699128Z 2024-08-20T22:38:31.7396401Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:38:31.7399326Z loading model: 0it [00:25, ?it/s] 2024-08-20T22:38:31.7429281Z cpu eval hf_GPT2_large 2024-08-20T22:38:31.7515588Z pass_due_to_skip 2024-08-20T22:38:31.7516293Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:38:31.7517015Z TIMING: 2024-08-20T22:38:31.7517286Z STATS: call_* op count: 0 2024-08-20T22:38:31.7518665Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:38:35.4207954Z 2024-08-20T22:38:51.2494299Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:38:51.2494762Z loading model: 0it [00:15, ?it/s] 2024-08-20T22:38:51.2524411Z cpu eval hf_T5_base 2024-08-20T22:39:19.5089547Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:39:30.2534702Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:39:41.0306658Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:42:49.4806601Z pass 2024-08-20T22:42:49.4809583Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:42:49.4811877Z TIMING: backend_compile:0.0004 entire_frame_compile:14.2028 total_wall_time:14.2028 2024-08-20T22:42:49.4813761Z STATS: call_* op count: 1492 | FakeTensorMode.__torch_dispatch__:84244 | FakeTensor.__torch_dispatch__:8503 | attempt fast:3332 | fast is_contiguous:3164 | slow both tensors nontrivially broadcast:24 | slow no contiguity match:144 | ProxyTorchDispatchMode.__torch_dispatch__:15624 2024-08-20T22:42:49.4815552Z Dynamo produced 1 graphs covering 1492 ops with 0 graph breaks (0 unique) 2024-08-20T22:42:55.8897852Z 2024-08-20T22:43:08.9386154Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:43:08.9386635Z loading model: 0it [00:13, ?it/s] 2024-08-20T22:43:08.9431123Z cpu eval hf_T5_large 2024-08-20T22:43:08.9431917Z pass_due_to_skip 2024-08-20T22:43:08.9433702Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:43:08.9436052Z TIMING: 2024-08-20T22:43:08.9436328Z STATS: call_* op count: 0 2024-08-20T22:43:08.9436874Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:43:12.3920611Z 2024-08-20T22:43:19.3660002Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:43:19.3660480Z loading model: 0it [00:06, ?it/s] 2024-08-20T22:43:19.3684879Z cpu eval hf_distil_whisper 2024-08-20T22:43:28.3265315Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:43:32.4388199Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:43:36.5551310Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:45:17.5062431Z /tmp/tmp14ep68yx/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/c3vycucyeswm5idzkufkimvhsrddj27athxjy64zzwehv2aspevm.cpp: In function ‘void cpp_fused_add_native_layer_norm_1(const float*, const float*, const float*, const float*, float*, float*, float*, long int)’: 2024-08-20T22:45:17.5071255Z /tmp/tmp14ep68yx/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/c3vycucyeswm5idzkufkimvhsrddj27athxjy64zzwehv2aspevm.cpp:513:40: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:45:17.5072870Z 513 | Welford tmp_acc0 = Welford(); 2024-08-20T22:45:17.5073400Z | ^~~~~~~~ 2024-08-20T22:45:17.5075533Z /tmp/tmp14ep68yx/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/c3vycucyeswm5idzkufkimvhsrddj27athxjy64zzwehv2aspevm.cpp: In function ‘void cpp_fused_add_native_layer_norm_3(const float*, const float*, const float*, const float*, const float*, float*, float*, float*, long int)’: 2024-08-20T22:45:17.5078606Z /tmp/tmp14ep68yx/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/c3vycucyeswm5idzkufkimvhsrddj27athxjy64zzwehv2aspevm.cpp:749:40: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:45:17.5080196Z 749 | Welford tmp_acc0 = Welford(); 2024-08-20T22:45:17.5080731Z | ^~~~~~~~ 2024-08-20T22:45:17.5083145Z /tmp/tmp14ep68yx/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/c3vycucyeswm5idzkufkimvhsrddj27athxjy64zzwehv2aspevm.cpp: In function ‘void cpp_fused_add_native_layer_norm_5(const float*, const float*, const float*, const float*, const float*, const float*, float*, float*, float*, long int)’: 2024-08-20T22:45:17.5086322Z /tmp/tmp14ep68yx/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/c3vycucyeswm5idzkufkimvhsrddj27athxjy64zzwehv2aspevm.cpp:957:40: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:45:17.5087901Z 957 | Welford tmp_acc0 = Welford(); 2024-08-20T22:45:17.5088438Z | ^~~~~~~~ 2024-08-20T22:46:11.1994857Z pass 2024-08-20T22:46:11.1997834Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:46:11.1999146Z TIMING: backend_compile:0.00021 entire_frame_compile:7.72013 total_wall_time:7.72013 2024-08-20T22:46:11.2001274Z STATS: call_* op count: 660 | FakeTensor.__torch_dispatch__:6829 | FakeTensorMode.__torch_dispatch__:65682 | attempt fast:5778 | fast is_contiguous:5337 | slow no contiguity match:441 | ProxyTorchDispatchMode.__torch_dispatch__:9016 2024-08-20T22:46:11.2003510Z Dynamo produced 1 graphs covering 660 ops with 0 graph breaks (0 unique) 2024-08-20T22:46:16.6529305Z 2024-08-20T22:46:16.7672663Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:46:16.7673150Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:46:16.7675764Z cpu eval lennard_jones 2024-08-20T22:46:16.7708695Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:46:16.7725451Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:46:16.7740619Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:46:36.1833266Z pass 2024-08-20T22:46:36.1834043Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:46:36.1835330Z TIMING: backend_compile:8e-05 entire_frame_compile:0.17649 total_wall_time:0.17649 2024-08-20T22:46:36.1836742Z STATS: call_* op count: 9 | FakeTensor.__torch_dispatch__:110 | FakeTensorMode.__torch_dispatch__:1255 | attempt fast:76 | fast is_contiguous:76 | ProxyTorchDispatchMode.__torch_dispatch__:218 2024-08-20T22:46:36.1838084Z Dynamo produced 1 graphs covering 9 ops with 0 graph breaks (0 unique) 2024-08-20T22:46:36.9322204Z accuracy pass_rate=57.14% 2024-08-20T22:46:36.9325480Z calls_captured gmean=0.00x mean=183.029x 2024-08-20T22:46:36.9329621Z unique_graphs gmean=0.00x mean=0.657x 2024-08-20T22:46:36.9333872Z graph_breaks gmean=0.00x mean=0.000x 2024-08-20T22:46:36.9337852Z unique_graph_breaks gmean=0.00x mean=0.000x 2024-08-20T22:46:36.9341578Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T22:46:36.9345185Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T22:46:36.9348878Z cudagraph_skips gmean=0.00x mean=0.000x 2024-08-20T22:46:38.7331060Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamic_cpu_aot_inductor_torchbench_freezing_inference.csv 2024-08-20T22:46:39.0684637Z BERT_pytorch XFAIL 2024-08-20T22:46:39.0685131Z Background_Matting XFAIL 2024-08-20T22:46:39.0688868Z LearningToPaint PASS 2024-08-20T22:46:39.0693638Z Super_SloMo PASS 2024-08-20T22:46:39.0698612Z alexnet PASS 2024-08-20T22:46:39.0703545Z basic_gnn_edgecnn PASS 2024-08-20T22:46:39.0708501Z basic_gnn_gcn PASS 2024-08-20T22:46:39.0713451Z basic_gnn_gin PASS 2024-08-20T22:46:39.0718432Z basic_gnn_sage XFAIL 2024-08-20T22:46:39.0723507Z dcgan PASS 2024-08-20T22:46:39.0728593Z demucs PASS 2024-08-20T22:46:39.0733577Z densenet121 PASS 2024-08-20T22:46:39.0738556Z detectron2_fasterrcnn_r_101_c4 XFAIL 2024-08-20T22:46:39.0743525Z detectron2_fasterrcnn_r_101_dc5 XFAIL 2024-08-20T22:46:39.0748476Z detectron2_fasterrcnn_r_101_fpn XFAIL 2024-08-20T22:46:39.0753446Z detectron2_fasterrcnn_r_50_c4 XFAIL 2024-08-20T22:46:39.0758401Z detectron2_fasterrcnn_r_50_dc5 XFAIL 2024-08-20T22:46:39.0763453Z detectron2_fasterrcnn_r_50_fpn XFAIL 2024-08-20T22:46:39.0769373Z dlrm XFAIL 2024-08-20T22:46:39.0774338Z doctr_det_predictor XFAIL 2024-08-20T22:46:39.0779291Z doctr_reco_predictor XFAIL 2024-08-20T22:46:39.0784250Z drq XFAIL 2024-08-20T22:46:39.0789511Z functorch_dp_cifar10 PASS 2024-08-20T22:46:39.0794231Z functorch_maml_omniglot PASS 2024-08-20T22:46:39.0799144Z hf_Albert PASS 2024-08-20T22:46:39.0804325Z hf_Bart PASS 2024-08-20T22:46:39.0809425Z hf_Bert PASS 2024-08-20T22:46:39.0814241Z hf_Bert_large PASS 2024-08-20T22:46:39.0819142Z hf_DistilBert PASS 2024-08-20T22:46:39.0824096Z hf_GPT2 PASS 2024-08-20T22:46:39.0829013Z hf_GPT2_large XFAIL 2024-08-20T22:46:39.0833921Z hf_T5_base PASS 2024-08-20T22:46:39.0838824Z hf_T5_large XFAIL 2024-08-20T22:46:39.0843941Z hf_distil_whisper PASS 2024-08-20T22:46:39.0848924Z lennard_jones PASS 2024-08-20T22:46:39.1234951Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamic_cpu_aot_inductor_torchbench_freezing_inference.csv 2024-08-20T22:46:39.4503278Z BERT_pytorch PASS 2024-08-20T22:46:39.4503809Z Background_Matting PASS 2024-08-20T22:46:39.4506863Z LearningToPaint PASS 2024-08-20T22:46:39.4511584Z Super_SloMo PASS 2024-08-20T22:46:39.4516421Z alexnet PASS 2024-08-20T22:46:39.4521151Z basic_gnn_edgecnn PASS 2024-08-20T22:46:39.4526206Z basic_gnn_gcn PASS 2024-08-20T22:46:39.4530937Z basic_gnn_gin PASS 2024-08-20T22:46:39.4535705Z basic_gnn_sage PASS 2024-08-20T22:46:39.4540423Z dcgan PASS 2024-08-20T22:46:39.4545145Z demucs PASS 2024-08-20T22:46:39.4549904Z densenet121 PASS 2024-08-20T22:46:39.4554677Z detectron2_fasterrcnn_r_101_c4 PASS 2024-08-20T22:46:39.4559488Z detectron2_fasterrcnn_r_101_dc5 PASS 2024-08-20T22:46:39.4564671Z detectron2_fasterrcnn_r_101_fpn PASS 2024-08-20T22:46:39.4569584Z detectron2_fasterrcnn_r_50_c4 PASS 2024-08-20T22:46:39.4574362Z detectron2_fasterrcnn_r_50_dc5 PASS 2024-08-20T22:46:39.4579137Z detectron2_fasterrcnn_r_50_fpn PASS 2024-08-20T22:46:39.4583857Z dlrm PASS 2024-08-20T22:46:39.4588604Z doctr_det_predictor PASS 2024-08-20T22:46:39.4593320Z doctr_reco_predictor PASS 2024-08-20T22:46:39.4598059Z drq PASS 2024-08-20T22:46:39.4602778Z functorch_dp_cifar10 PASS 2024-08-20T22:46:39.4607758Z functorch_maml_omniglot PASS 2024-08-20T22:46:39.4612448Z hf_Albert PASS 2024-08-20T22:46:39.4617223Z hf_Bart PASS 2024-08-20T22:46:39.4621979Z hf_Bert PASS 2024-08-20T22:46:39.4626732Z hf_Bert_large PASS 2024-08-20T22:46:39.4631458Z hf_DistilBert PASS 2024-08-20T22:46:39.4636201Z hf_GPT2 PASS 2024-08-20T22:46:39.4640950Z hf_GPT2_large PASS 2024-08-20T22:46:39.4646015Z hf_T5_base PASS 2024-08-20T22:46:39.4650791Z hf_T5_large PASS 2024-08-20T22:46:39.4655460Z hf_distil_whisper PASS 2024-08-20T22:46:39.4660210Z lennard_jones PASS 2024-08-20T22:46:39.5046427Z + cleanup_workspace 2024-08-20T22:46:39.5048450Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T22:46:39.5049585Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T22:46:39.5050569Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T22:46:39.5051472Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T22:46:39.5052299Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T22:46:39.5053207Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T22:46:39.5053912Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T22:46:40.1443169Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T22:46:40.1443686Z cat test/**/*_toprint.log || true 2024-08-20T22:46:40.1467501Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:40.1467985Z env: 2024-08-20T22:46:40.1468251Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:40.1468870Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:40.1469536Z ##[endgroup] 2024-08-20T22:46:40.1541596Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T22:46:40.1570576Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T22:46:40.1571028Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T22:46:40.1576984Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:40.1577463Z env: 2024-08-20T22:46:40.1577700Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:40.1578394Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:40.1579019Z MONITOR_SCRIPT_PID: 363902 2024-08-20T22:46:40.1579322Z ##[endgroup] 2024-08-20T22:46:40.1788241Z Prepare all required actions 2024-08-20T22:46:40.1788653Z Getting action download info 2024-08-20T22:46:40.3796595Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T22:46:40.5525848Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T22:46:40.5526612Z with: 2024-08-20T22:46:40.5527668Z file-suffix: test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430 2024-08-20T22:46:40.5528859Z s3-bucket: gha-artifacts 2024-08-20T22:46:40.5529290Z env: 2024-08-20T22:46:40.5529684Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:40.5530633Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:40.5531701Z ##[endgroup] 2024-08-20T22:46:40.5561828Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T22:46:40.5562429Z # Remove any previous test jsons if they exist 2024-08-20T22:46:40.5563076Z rm -f test-jsons-*.zip 2024-08-20T22:46:40.5563742Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T22:46:40.5569740Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:40.5570234Z env: 2024-08-20T22:46:40.5570502Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:40.5571100Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:40.5572126Z FILE_SUFFIX: test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430 2024-08-20T22:46:40.5572858Z ##[endgroup] 2024-08-20T22:46:40.5806221Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T22:46:40.5844160Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T22:46:40.5863595Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T22:46:40.5882571Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T22:46:40.5902159Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T22:46:40.5928167Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T22:46:40.5928804Z # Remove any previous test reports if they exist 2024-08-20T22:46:40.5929321Z rm -f test-reports-*.zip 2024-08-20T22:46:40.5929897Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T22:46:40.5935472Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:40.5936038Z env: 2024-08-20T22:46:40.5936282Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:40.5936851Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:40.5937791Z FILE_SUFFIX: test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430 2024-08-20T22:46:40.5938453Z ##[endgroup] 2024-08-20T22:46:40.6146378Z adding: test/test-reports/inference_torchbench.csv (deflated 71%) 2024-08-20T22:46:40.6172700Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T22:46:40.6173343Z # Remove any previous usage logs if they exist 2024-08-20T22:46:40.6173834Z rm -f logs-*.zip 2024-08-20T22:46:40.6174916Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T22:46:40.6176388Z # so check to see if the file exists first 2024-08-20T22:46:40.6177311Z if [ -f 'usage_log.txt' ]; then 2024-08-20T22:46:40.6178385Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T22:46:40.6179076Z fi 2024-08-20T22:46:40.6179631Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T22:46:40.6180701Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T22:46:40.6181621Z fi 2024-08-20T22:46:40.6187680Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:40.6188159Z env: 2024-08-20T22:46:40.6188400Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:40.6189232Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:40.6190169Z FILE_SUFFIX: test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430 2024-08-20T22:46:40.6190846Z ##[endgroup] 2024-08-20T22:46:40.6258922Z adding: usage_log.txt (deflated 92%) 2024-08-20T22:46:40.6327842Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T22:46:40.6328552Z # Remove any previous debugging artifacts if they exist 2024-08-20T22:46:40.6329104Z rm -f debug-*.zip 2024-08-20T22:46:40.6329464Z if [ -d 'test/debug' ]; then 2024-08-20T22:46:40.6329940Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T22:46:40.6330399Z fi 2024-08-20T22:46:40.6335365Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:40.6335918Z env: 2024-08-20T22:46:40.6336160Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:40.6336729Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:40.6337663Z FILE_SUFFIX: test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430 2024-08-20T22:46:40.6338326Z ##[endgroup] 2024-08-20T22:46:40.6417222Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:46:40.6417628Z with: 2024-08-20T22:46:40.6417884Z s3-bucket: gha-artifacts 2024-08-20T22:46:40.6418291Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:46:40.6418753Z retention-days: 14 2024-08-20T22:46:40.6419047Z if-no-files-found: warn 2024-08-20T22:46:40.6419375Z path: test-jsons-*.zip 2024-08-20T22:46:40.6419684Z name: artifact 2024-08-20T22:46:40.6419959Z region: us-east-1 2024-08-20T22:46:40.6420217Z env: 2024-08-20T22:46:40.6420462Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:40.6421045Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:40.6421668Z ##[endgroup] 2024-08-20T22:46:40.9933225Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:46:40.9933853Z With the provided path, there will be 1 file uploaded 2024-08-20T22:46:40.9934471Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:46:40.9972572Z Starting upload of test-jsons-test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430.zip 2024-08-20T22:46:41.0892500Z Finished upload of test-jsons-test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430.zip 2024-08-20T22:46:41.1092076Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:46:41.1092506Z with: 2024-08-20T22:46:41.1092776Z s3-bucket: gha-artifacts 2024-08-20T22:46:41.1093205Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:46:41.1093669Z retention-days: 14 2024-08-20T22:46:41.1093977Z if-no-files-found: error 2024-08-20T22:46:41.1094332Z path: test-reports-*.zip 2024-08-20T22:46:41.1094666Z name: artifact 2024-08-20T22:46:41.1094938Z region: us-east-1 2024-08-20T22:46:41.1095222Z env: 2024-08-20T22:46:41.1095488Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:41.1096164Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:41.1096760Z ##[endgroup] 2024-08-20T22:46:41.4371532Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:46:41.4372156Z With the provided path, there will be 1 file uploaded 2024-08-20T22:46:41.4372777Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:46:41.4410236Z Starting upload of test-reports-test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430.zip 2024-08-20T22:46:41.5425910Z Finished upload of test-reports-test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430.zip 2024-08-20T22:46:41.5626847Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:46:41.5627238Z with: 2024-08-20T22:46:41.5627485Z s3-bucket: gha-artifacts 2024-08-20T22:46:41.5627879Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:46:41.5628473Z retention-days: 14 2024-08-20T22:46:41.5628760Z if-no-files-found: ignore 2024-08-20T22:46:41.5629081Z path: logs-*.zip 2024-08-20T22:46:41.5629351Z name: artifact 2024-08-20T22:46:41.5629603Z region: us-east-1 2024-08-20T22:46:41.5629866Z env: 2024-08-20T22:46:41.5630107Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:41.5630667Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:41.5631272Z ##[endgroup] 2024-08-20T22:46:41.8918872Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:46:41.8919487Z With the provided path, there will be 1 file uploaded 2024-08-20T22:46:41.8920089Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:46:41.8956589Z Starting upload of logs-test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430.zip 2024-08-20T22:46:42.0137585Z Finished upload of logs-test-dynamic_cpu_aot_inductor_torchbench_freezing-1-2-amz2023.linux.12xlarge_29025551430.zip 2024-08-20T22:46:42.0333703Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:46:42.0334102Z with: 2024-08-20T22:46:42.0334357Z s3-bucket: gha-artifacts 2024-08-20T22:46:42.0334761Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:46:42.0335201Z retention-days: 14 2024-08-20T22:46:42.0335486Z if-no-files-found: ignore 2024-08-20T22:46:42.0335812Z path: debug-*.zip 2024-08-20T22:46:42.0336101Z name: artifact 2024-08-20T22:46:42.0336353Z region: us-east-1 2024-08-20T22:46:42.0336621Z env: 2024-08-20T22:46:42.0336859Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:42.0337424Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:42.0338022Z ##[endgroup] 2024-08-20T22:46:42.3554658Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T22:46:42.3749962Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T22:46:42.3750400Z # shellcheck disable=SC2156 2024-08-20T22:46:42.3751145Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T22:46:42.3756880Z shell: /usr/bin/bash -e {0} 2024-08-20T22:46:42.3757199Z env: 2024-08-20T22:46:42.3757440Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:42.3758003Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:42.3758609Z ##[endgroup] 2024-08-20T22:46:42.5942347Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T22:46:42.5942894Z with: 2024-08-20T22:46:42.5943128Z env: 2024-08-20T22:46:42.5943375Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:42.5944037Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:42.5944652Z ##[endgroup] 2024-08-20T22:46:42.5983412Z ##[group]Run set -eou pipefail 2024-08-20T22:46:42.5983769Z set -eou pipefail 2024-08-20T22:46:42.5984064Z  2024-08-20T22:46:42.5984714Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T22:46:42.5985323Z for _ in $(seq 1440); do 2024-08-20T22:46:42.5985761Z  # Break if no ssh session exists anymore 2024-08-20T22:46:42.5986218Z  if [ "$(who)" = "" ]; then 2024-08-20T22:46:42.5986694Z  break 2024-08-20T22:46:42.5986964Z  fi 2024-08-20T22:46:42.5987251Z  echo "." 2024-08-20T22:46:42.5987653Z  sleep 5 2024-08-20T22:46:42.5987926Z done 2024-08-20T22:46:42.5992966Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:42.5993408Z env: 2024-08-20T22:46:42.5993643Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:42.5994200Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:42.5994804Z ##[endgroup] 2024-08-20T22:46:42.6016356Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T22:46:42.6082797Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:46:42.6083793Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:46:42.6084377Z # shellcheck disable=SC2046 2024-08-20T22:46:42.6084819Z docker stop $(docker ps -q) || true 2024-08-20T22:46:42.6085270Z # Prune all of the docker images 2024-08-20T22:46:42.6085704Z docker system prune -af 2024-08-20T22:46:42.6091032Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:42.6091487Z env: 2024-08-20T22:46:42.6091725Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:42.6092293Z DOCKER_CONTAINER_ID: b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:42.6092893Z ##[endgroup] 2024-08-20T22:46:53.3312254Z b5c5ea11556a 2024-08-20T22:46:54.8987326Z Deleted Containers: 2024-08-20T22:46:54.8988070Z b5c5ea11556ac1b266d307df39bb18a9b42c90996130019b9d6c522efcadd7ab 2024-08-20T22:46:54.8988538Z 2024-08-20T22:46:58.8007646Z Deleted Images: 2024-08-20T22:46:58.8009792Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:46:58.8012138Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks@sha256:ed18f711327455e269271a17448384275684348a472ba289f00904bd14501fa9 2024-08-20T22:46:58.8013664Z deleted: sha256:822356886a68936cce34917f5ffc60d864918fab7594bb08c26bf1816dd6110d 2024-08-20T22:46:58.8014564Z deleted: sha256:ae6b018f5c327102b48941d39cd8d6fa6151893715290829a76e4e73add3a418 2024-08-20T22:46:58.8015451Z deleted: sha256:79fa7dddbf1ed3af5e8f10972d07ccea774929efe8578a1cc6604a00c93c1e02 2024-08-20T22:46:58.8016331Z deleted: sha256:06b439d80ae875bb9b96d0a23c4619aec5b5fb2f35b5048022ebe6061e19f7e1 2024-08-20T22:46:58.8017197Z deleted: sha256:52d3d63b327ae120e60f76fe27965e31638eb8d556b5e5f56f4773bd940373cf 2024-08-20T22:46:58.8018050Z deleted: sha256:5df173ad6b0f1ba16287832b6e5237513d152e87e627e478fc36ed16159697c0 2024-08-20T22:46:58.8018911Z deleted: sha256:b6c13bf75251ea08468f4040dbba32848f72cd00eb8c3003ecc1e879f73738ae 2024-08-20T22:46:58.8019765Z deleted: sha256:f2f37b6c898fc6a8c74683cc50a156f56abe71f20310c72338bb0ec23c71e8d6 2024-08-20T22:46:58.8020648Z deleted: sha256:55832a6aecdb11a84ddbbd38efcac3b62dd61a2b7832a1080535fc5730e1e336 2024-08-20T22:46:58.8021917Z deleted: sha256:bd86c04ac5f8ba1940bc991dfbae9e17a80293b5cf5fe568c694f46d59863011 2024-08-20T22:46:58.8022809Z deleted: sha256:88a76e0add806e67ed7bfd8ad7f862a8683d087dd3557cfc2b787da5f4c1ae0f 2024-08-20T22:46:58.8023824Z deleted: sha256:37a2749f2e5a556bb0369b3db12c47e4b82b41064f430896babceb49d97d2f9f 2024-08-20T22:46:58.8024669Z deleted: sha256:ac6214a417e93e76048bee154136a3117823ef38824b38f2659c74c7d09b57f8 2024-08-20T22:46:58.8025541Z deleted: sha256:bd60508bf2fcb98670e5ba90896321868f6fb9c5a2c859203e47512542f45b99 2024-08-20T22:46:58.8026855Z deleted: sha256:3fe3c9441ff3190f5cf87c066d3b5ca03aa3bb68071ef86e127c271ebb1ea02e 2024-08-20T22:46:58.8027697Z deleted: sha256:f9b8978a3de27376176f0400dcb17127771f2a8123ff323474e055dbae6eaba5 2024-08-20T22:46:58.8028540Z deleted: sha256:825e6cc758ce8478a096de8d367123d4d0a7f51fbe595867d2f950f714ca908e 2024-08-20T22:46:58.8029399Z deleted: sha256:dde33dfb74ab43131553ba775cd8561874596ba2e3d1b810c7796beba3d4fd38 2024-08-20T22:46:58.8030261Z deleted: sha256:809173faaa7c1289371a0c5a0927a6050de23417cddba0746dea79b10b68dd71 2024-08-20T22:46:58.8031248Z deleted: sha256:f63fee6ae14582933c75117b8d3115041d0177792e510d8dab0baca1b2202771 2024-08-20T22:46:58.8032113Z deleted: sha256:9432a4a5f47ddff6148fd5f8ade43d3a5566c0da2fbcd904daa917dc21c4c73d 2024-08-20T22:46:58.8033002Z deleted: sha256:b9ff51a3d1dc73e8a3ab751fa1a512026a101eaedbea74cfe01a2876b5111502 2024-08-20T22:46:58.8033865Z deleted: sha256:70e4013b779cc78a919225cad0a55d38948b0620ea119eb80dbb377ce8687f7e 2024-08-20T22:46:58.8034709Z deleted: sha256:edcf423609137384d678c9f677cae55933016f3ee76cc2ee47d93aad1346c427 2024-08-20T22:46:58.8035552Z deleted: sha256:bb2f6a3359e809db17e8b77499bf0d94200af9defc470b3adb3e1c28a6f1c6c1 2024-08-20T22:46:58.8036404Z deleted: sha256:eb646c0b0638f760f7c1046a113d3646e57513d19390881f4f6775ec702fa265 2024-08-20T22:46:58.8037359Z deleted: sha256:71c5137ede0421f7f00b1e93ee7bd6287c8ff6445615d0e1d1edddff2240789c 2024-08-20T22:46:58.8038184Z deleted: sha256:d2a20794246965e241eaaa98883ecd02a39901618f06536532bb4d9ed2b80f63 2024-08-20T22:46:58.8039179Z deleted: sha256:a8fc5137968c5840b35ff322f4796356303c4d4f0ed5c86c70218e20045936ac 2024-08-20T22:46:58.8040021Z deleted: sha256:1bd367d8cc65cb8808ddf7665064a0523467c7947ead2e1aed5112e9465e2fa8 2024-08-20T22:46:58.8040872Z deleted: sha256:736b605a3af6c5f2f82b517f0c1c517701a78def165ad99f3fb034198818994d 2024-08-20T22:46:58.8041839Z deleted: sha256:f2766fd6c51e0f9c3fdeed91ff144d049ed99f1a2cdce5c3b36a9f694af46045 2024-08-20T22:46:58.8042697Z deleted: sha256:688dce500613b1366475010344298a9b4eca441474fb871cfc37287a6eefbbf4 2024-08-20T22:46:58.8043616Z deleted: sha256:08f9d176a447289b58f9c70cdc76546487fc2700b1e5a7b79c19eb4f8f7c4608 2024-08-20T22:46:58.8044485Z deleted: sha256:538a2fc18bbc7da30a73c02bc8bb61bd6800a9d501caeaec614a32c41d53438d 2024-08-20T22:46:58.8045360Z deleted: sha256:75d7a20f4c66e80cf12fdf6d68787ee75c33478fd7c3e30c4696702ef9932993 2024-08-20T22:46:58.8046209Z deleted: sha256:5f36a35d97f9c931857392a1a616a5af4da00dfabe321b21f7637761d680b575 2024-08-20T22:46:58.8047061Z deleted: sha256:172cb1246a46e9dde7d50405f7fb31fd83a3a3e6ff40c811cfe100bd034807f6 2024-08-20T22:46:58.8047930Z deleted: sha256:d621ca4d3a5a15f4d74797b18bd0f4c44f63288c18e753fc4281cec69d82ea21 2024-08-20T22:46:58.8048795Z deleted: sha256:8e5eec944591edb185a54d6b495949dcae1da0be3fec8d30c3463e9fcde1a34e 2024-08-20T22:46:58.8049654Z deleted: sha256:656151b8c659bb4dab88293de022523e0e478e654f6922eaf53efb36ffb26e7e 2024-08-20T22:46:58.8050476Z deleted: sha256:9e2c8ec2c392f561f1016a988a64620f377203ed895815f024856cdef934be6b 2024-08-20T22:46:58.8051323Z deleted: sha256:c42f2e17dc37198a178852ecf9d947775b76af99ebdc2aa929af4b03b3896b3b 2024-08-20T22:46:58.8052177Z deleted: sha256:f52f63d8f16c18cde14e5738947c6e08323452e8599f723a6e041ed1ab33565a 2024-08-20T22:46:58.8053034Z deleted: sha256:343dc9f9b3e3c25dc528bbe1f8b4258289e512ff85f4b9c22ea2154c9c9d8143 2024-08-20T22:46:58.8053890Z deleted: sha256:5efa5da92a100a8c36c51a732af9534a8023041baf9d55ecf831c1386467d65a 2024-08-20T22:46:58.8054851Z deleted: sha256:da5cd90133fe64eb015b47ab4515d89305598acefc5c097b8e189da64be2d476 2024-08-20T22:46:58.8055715Z deleted: sha256:691e04884c858a1ea0a97381a7b1036bdfc6f9a48471608f0a5db9da9935555f 2024-08-20T22:46:58.8056567Z deleted: sha256:64100fb171394a71add5752ea3f6c432bcba0ac31aca67d8443c3722864804e1 2024-08-20T22:46:58.8057410Z deleted: sha256:a03197ec2ed41935ef46795a87ffd3d80b526fce1edf28550177f579cfb5df64 2024-08-20T22:46:58.8058265Z deleted: sha256:7bd9c2008289af25e772b1fc7b8e6470b513a5f76ce25743b9ef1bbe43ace5f7 2024-08-20T22:46:58.8059118Z deleted: sha256:0064821cdad60c711279a5964f88bf82c7582c296d21a0d1d0e97c60a47c1fa3 2024-08-20T22:46:58.8059964Z deleted: sha256:5825eb9623523140ad1e3ad683f3a83ab39054adf0b0fb4e472c2a7c3e7545ff 2024-08-20T22:46:58.8060827Z deleted: sha256:ff2d63ec14fa462e2edcd48d8ca8b678d79c9981bda17880818db15b8790e54b 2024-08-20T22:46:58.8061689Z deleted: sha256:79a5278d4bd098095d2f18fcfa476e8d54112dae4ac3d6c4c955e7fd9ba550f4 2024-08-20T22:46:58.8062532Z deleted: sha256:808ef6924cb221e2093f74413950d96ff62707fc13a194d83a0422315fccccdc 2024-08-20T22:46:58.8063489Z deleted: sha256:91aa45c12c4226ea9961e3acd079df4454bf2a55cb1bb9a2918f37bec89947bf 2024-08-20T22:46:58.8064534Z deleted: sha256:1f26a67abbf110b4883c6dd39e035f74248d28916c4a48be34296a7a96d1319b 2024-08-20T22:46:58.8065378Z deleted: sha256:1165de789bc53191b61c2b04dc04cf9d3d6d78f3d704776bd5a276d2caac8d09 2024-08-20T22:46:58.8066263Z deleted: sha256:aff63be2ff8d27d7172fb6dbecff0bea191a85c2431f8de65c0f81d40212587b 2024-08-20T22:46:58.8067149Z deleted: sha256:7c4290bd5a95669ca4d362a8672cb96b80693cc40af13f1ce061e86f835766d1 2024-08-20T22:46:58.8068014Z deleted: sha256:46ad9cfd3aa9b54d845ebc37c2e4c0d5fda90577227a7962049b71c825acdafe 2024-08-20T22:46:58.8068882Z deleted: sha256:aad12f955a8ca5766f6ca5d1467a6d2b81be713ff7224e8ae8eb92b68acf69a2 2024-08-20T22:46:58.8069755Z deleted: sha256:f5a053a7d9bda27126802be2ba52d187c26e934b7cc582850d2aaadd90631a1e 2024-08-20T22:46:58.8070615Z deleted: sha256:cf0f3c2eb852401868a3f959f6176f5a6b42f17f16e950f132f03293c764d733 2024-08-20T22:46:58.8071439Z deleted: sha256:7929c61352934651542ff23122529355349b2153871117dcfbaa4b45020a158d 2024-08-20T22:46:58.8072261Z deleted: sha256:119390db59d077c75caa11c49fee6ab586d061d27de1deeeec8608f330624f21 2024-08-20T22:46:58.8073114Z deleted: sha256:3b7424223daa8c5790ef526174f3780d9b1b54baa45d080127bfc72e60139a2b 2024-08-20T22:46:58.8073958Z deleted: sha256:f8986b3dfad5395410b72709f00789a62ee60d1b0bc1b27583f4c17af34b43fc 2024-08-20T22:46:58.8074801Z deleted: sha256:03db7655e84b428b1399087a9ed34180c120808cd00d94924a46eeaef73c37b0 2024-08-20T22:46:58.8075641Z deleted: sha256:b4a81bb8166fcf8ffbf063817b3202900c20586ca4bf369b21822764ab966164 2024-08-20T22:46:58.8076609Z deleted: sha256:afc241c47ba97b4a4f054cd0e6a5d90428b2675d00b160727187b7feabb81b14 2024-08-20T22:46:58.8077454Z deleted: sha256:e04730e6f5fc53e139356efef9c2c84b7bf6087c5a89876deaf1be58b9b1029b 2024-08-20T22:46:58.8078302Z deleted: sha256:7b22c8a2bc4bb0051ddc3ea51c917e7562663e4dafe4f9ad4a3870f4ec84bd14 2024-08-20T22:46:58.8079148Z deleted: sha256:c54b28287358751c9e19bd4f52b1d920ec5f4a254f8883b9d093528c08bc8b3f 2024-08-20T22:46:58.8079965Z deleted: sha256:17303ef9705fd22b570529dc8413ccba1a59fab7f5fef103bfe72e01c53d976e 2024-08-20T22:46:58.8080813Z deleted: sha256:1a4e7403586e712f2caf32f8bec553dafd88f78612fba5bd0c54e4efb89b9c41 2024-08-20T22:46:58.8081664Z deleted: sha256:d2f2462c603417aa43b82ea6f370ed4319cd12b3bda370d5a7ada0845daf3aaa 2024-08-20T22:46:58.8082512Z deleted: sha256:973bcd7518ef8f8e81b3b895b1ff4fe6e8a20915387d1f127edefccdcc3d867d 2024-08-20T22:46:58.8083385Z deleted: sha256:ab913355e8147522ebc4f29618812090844aa39956668afc4b91926827c327b3 2024-08-20T22:46:58.8084387Z deleted: sha256:69527fc1954cb8778833d15c0f3562fda1cbd230b11e649a974752f7a726cbf9 2024-08-20T22:46:58.8085236Z deleted: sha256:1b2a52846c812d7caf91918531cb04230500b31c3f695e444c1885aad0777b71 2024-08-20T22:46:58.8086092Z deleted: sha256:40f793cdc6b0a26ec9af5ea7188ad0ef19799a234e53eb4b8f3f756694a1000e 2024-08-20T22:46:58.8086952Z deleted: sha256:39b098ef95425e2d1bdb44559eda25dfc7bd62af2fb6bb691b540741278acb95 2024-08-20T22:46:58.8087959Z deleted: sha256:bc116c6279a58082c4ef9dac27e9d4d76bfdd78c2f2fe4439663d3dd2af55b57 2024-08-20T22:46:58.8088825Z deleted: sha256:57046a92505d1125e1af1100aaeb0e36175039b7dd36e67e7b4c554997d81b07 2024-08-20T22:46:58.8089756Z deleted: sha256:e868a8284f40af2175a17211327660844c1ac66ee31919f52c5e4e2b9564213a 2024-08-20T22:46:58.8090537Z deleted: sha256:ba5f7b6add2ae68d2962f19c3d981adc39633683af6dc3392c354cce87b7e8f9 2024-08-20T22:46:58.8091327Z deleted: sha256:3a06302b80c87d1030f701f23861d185613a3a4269faf533bb1b82b092a82887 2024-08-20T22:46:58.8092115Z deleted: sha256:dbdee317045b3156074f52d1fb32d613a48b6d8ed13c3e400a9f2c7bcceaf808 2024-08-20T22:46:58.8092927Z deleted: sha256:9574f911b87c39bbbb08a0238b82fb53abdedad1f387beb999ce1d36f3845e98 2024-08-20T22:46:58.8093743Z deleted: sha256:15d49efcad797dff0aa23e3b025b50c1c961668637e4bd9dff2361eaf3405c74 2024-08-20T22:46:58.8094551Z deleted: sha256:1b9b7346fee7abbc7f5538eaa23548bd05a45abe8daf6794024be0c8ad7d60bb 2024-08-20T22:46:58.8095126Z 2024-08-20T22:46:58.8095259Z Total reclaimed space: 35.51GB 2024-08-20T22:46:58.8159173Z Post job cleanup. 2024-08-20T22:46:58.8207231Z Post job cleanup. 2024-08-20T22:46:58.9055304Z [command]/usr/bin/git version 2024-08-20T22:46:58.9112109Z git version 2.40.1 2024-08-20T22:46:58.9150026Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/82cc8f64-c33b-4c03-9ed3-6316d8f2a1b7' before making global git config changes 2024-08-20T22:46:58.9151333Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:46:58.9153240Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:46:58.9189872Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:46:58.9222752Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:46:58.9541248Z Entering 'android/libs/fbjni' 2024-08-20T22:46:58.9593559Z Entering 'third_party/FP16' 2024-08-20T22:46:58.9644647Z Entering 'third_party/FXdiv' 2024-08-20T22:46:58.9694663Z Entering 'third_party/NNPACK' 2024-08-20T22:46:58.9746344Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:46:58.9797431Z Entering 'third_party/XNNPACK' 2024-08-20T22:46:58.9865712Z Entering 'third_party/benchmark' 2024-08-20T22:46:58.9917878Z Entering 'third_party/cpp-httplib' 2024-08-20T22:46:58.9967174Z Entering 'third_party/cpuinfo' 2024-08-20T22:46:59.0019584Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:46:59.0070755Z Entering 'third_party/cutlass' 2024-08-20T22:46:59.0129923Z Entering 'third_party/eigen' 2024-08-20T22:46:59.0186060Z Entering 'third_party/fbgemm' 2024-08-20T22:46:59.0237615Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:46:59.0290014Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:46:59.0340533Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:46:59.0397880Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:46:59.0446124Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:46:59.0501699Z Entering 'third_party/flatbuffers' 2024-08-20T22:46:59.0561357Z Entering 'third_party/fmt' 2024-08-20T22:46:59.0619441Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:46:59.0668687Z Entering 'third_party/gloo' 2024-08-20T22:46:59.0720065Z Entering 'third_party/googletest' 2024-08-20T22:46:59.0771140Z Entering 'third_party/ideep' 2024-08-20T22:46:59.0821188Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:46:59.0881354Z Entering 'third_party/ittapi' 2024-08-20T22:46:59.0930500Z Entering 'third_party/kineto' 2024-08-20T22:46:59.0982619Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:46:59.1032091Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:46:59.1084185Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:46:59.1132460Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:46:59.1183736Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:46:59.1232814Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:46:59.1285208Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:46:59.1340176Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:46:59.1390323Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:46:59.1441060Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:46:59.1490785Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:46:59.1542530Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:46:59.1596845Z Entering 'third_party/mimalloc' 2024-08-20T22:46:59.1645305Z Entering 'third_party/nccl/nccl' 2024-08-20T22:46:59.1698061Z Entering 'third_party/nlohmann' 2024-08-20T22:46:59.1748878Z Entering 'third_party/onnx' 2024-08-20T22:46:59.1815034Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:46:59.1865635Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:46:59.1925242Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:46:59.1982069Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:46:59.2032098Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:46:59.2082241Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:46:59.2128942Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:46:59.2181579Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:46:59.2231021Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:46:59.2287891Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:46:59.2340099Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:46:59.2392262Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:46:59.2444037Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:46:59.2514745Z Entering 'third_party/pocketfft' 2024-08-20T22:46:59.2564993Z Entering 'third_party/protobuf' 2024-08-20T22:46:59.2620821Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:46:59.2669291Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:46:59.2724267Z Entering 'third_party/psimd' 2024-08-20T22:46:59.2778680Z Entering 'third_party/pthreadpool' 2024-08-20T22:46:59.2830906Z Entering 'third_party/pybind11' 2024-08-20T22:46:59.2882463Z Entering 'third_party/python-peachpy' 2024-08-20T22:46:59.2931416Z Entering 'third_party/sleef' 2024-08-20T22:46:59.2983212Z Entering 'third_party/tensorpipe' 2024-08-20T22:46:59.3034206Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:46:59.3084147Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:46:59.3131570Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:46:59.3181862Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:46:59.3229655Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:46:59.3301946Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:46:59.3330799Z http.https://github.com/.extraheader 2024-08-20T22:46:59.3338582Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T22:46:59.3371969Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:46:59.3659372Z Entering 'android/libs/fbjni' 2024-08-20T22:46:59.3692570Z http.https://github.com/.extraheader 2024-08-20T22:46:59.3727406Z Entering 'third_party/FP16' 2024-08-20T22:46:59.3762339Z http.https://github.com/.extraheader 2024-08-20T22:46:59.3795962Z Entering 'third_party/FXdiv' 2024-08-20T22:46:59.3829227Z http.https://github.com/.extraheader 2024-08-20T22:46:59.3864732Z Entering 'third_party/NNPACK' 2024-08-20T22:46:59.3898485Z http.https://github.com/.extraheader 2024-08-20T22:46:59.3931853Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:46:59.3965246Z http.https://github.com/.extraheader 2024-08-20T22:46:59.3998475Z Entering 'third_party/XNNPACK' 2024-08-20T22:46:59.4033969Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4083768Z Entering 'third_party/benchmark' 2024-08-20T22:46:59.4118954Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4150749Z Entering 'third_party/cpp-httplib' 2024-08-20T22:46:59.4185000Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4218787Z Entering 'third_party/cpuinfo' 2024-08-20T22:46:59.4250931Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4285026Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:46:59.4318109Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4350268Z Entering 'third_party/cutlass' 2024-08-20T22:46:59.4384572Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4431367Z Entering 'third_party/eigen' 2024-08-20T22:46:59.4465471Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4499746Z Entering 'third_party/fbgemm' 2024-08-20T22:46:59.4532256Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4565615Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:46:59.4599736Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4632542Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:46:59.4665780Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4698185Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:46:59.4730022Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4771578Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:46:59.4804925Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4837061Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:46:59.4869808Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4904471Z Entering 'third_party/flatbuffers' 2024-08-20T22:46:59.4937910Z http.https://github.com/.extraheader 2024-08-20T22:46:59.4971932Z Entering 'third_party/fmt' 2024-08-20T22:46:59.5005114Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5038248Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:46:59.5071147Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5104389Z Entering 'third_party/gloo' 2024-08-20T22:46:59.5137595Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5169565Z Entering 'third_party/googletest' 2024-08-20T22:46:59.5204342Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5238159Z Entering 'third_party/ideep' 2024-08-20T22:46:59.5275486Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5306281Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:46:59.5339547Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5382999Z Entering 'third_party/ittapi' 2024-08-20T22:46:59.5419866Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5451437Z Entering 'third_party/kineto' 2024-08-20T22:46:59.5485200Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5522299Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:46:59.5556327Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5588770Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:46:59.5623202Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5665719Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:46:59.5700816Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5741011Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:46:59.5774855Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5809453Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:46:59.5843492Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5882637Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:46:59.5915622Z http.https://github.com/.extraheader 2024-08-20T22:46:59.5958859Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:46:59.5994832Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6027995Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:46:59.6061232Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6095072Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:46:59.6127668Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6170553Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:46:59.6204883Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6238646Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:46:59.6270956Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6302544Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:46:59.6333410Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6370237Z Entering 'third_party/mimalloc' 2024-08-20T22:46:59.6404617Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6436831Z Entering 'third_party/nccl/nccl' 2024-08-20T22:46:59.6468901Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6501018Z Entering 'third_party/nlohmann' 2024-08-20T22:46:59.6533099Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6566141Z Entering 'third_party/onnx' 2024-08-20T22:46:59.6601795Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6652770Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:46:59.6686716Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6719692Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:46:59.6755241Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6796232Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:46:59.6830170Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6863321Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:46:59.6894896Z http.https://github.com/.extraheader 2024-08-20T22:46:59.6927865Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:46:59.6961461Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7000069Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:46:59.7034033Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7066035Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:46:59.7099812Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7132611Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:46:59.7165941Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7201617Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:46:59.7234355Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7265790Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:46:59.7300585Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7331390Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:46:59.7364948Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7398871Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:46:59.7431660Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7465625Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:46:59.7499380Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7552774Z Entering 'third_party/pocketfft' 2024-08-20T22:46:59.7586788Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7623874Z Entering 'third_party/protobuf' 2024-08-20T22:46:59.7655722Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7691834Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:46:59.7724930Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7760327Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:46:59.7794510Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7827317Z Entering 'third_party/psimd' 2024-08-20T22:46:59.7862408Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7896657Z Entering 'third_party/pthreadpool' 2024-08-20T22:46:59.7928191Z http.https://github.com/.extraheader 2024-08-20T22:46:59.7965134Z Entering 'third_party/pybind11' 2024-08-20T22:46:59.7998486Z http.https://github.com/.extraheader 2024-08-20T22:46:59.8029243Z Entering 'third_party/python-peachpy' 2024-08-20T22:46:59.8063583Z http.https://github.com/.extraheader 2024-08-20T22:46:59.8102008Z Entering 'third_party/sleef' 2024-08-20T22:46:59.8135815Z http.https://github.com/.extraheader 2024-08-20T22:46:59.8167796Z Entering 'third_party/tensorpipe' 2024-08-20T22:46:59.8201682Z http.https://github.com/.extraheader 2024-08-20T22:46:59.8237841Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:46:59.8269280Z http.https://github.com/.extraheader 2024-08-20T22:46:59.8301082Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:46:59.8334524Z http.https://github.com/.extraheader 2024-08-20T22:46:59.8365376Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:46:59.8400501Z http.https://github.com/.extraheader 2024-08-20T22:46:59.8432375Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:46:59.8465596Z http.https://github.com/.extraheader 2024-08-20T22:46:59.8494650Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:46:59.8526671Z http.https://github.com/.extraheader 2024-08-20T22:46:59.8651513Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T22:46:59.8668625Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T22:46:59.8673954Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:59.8674451Z ##[endgroup] 2024-08-20T22:47:06.1977960Z Cleaning up orphan processes