2024-08-20T22:04:46.2120899Z Current runner version: '2.319.1' 2024-08-20T22:04:46.2127254Z Runner name: 'i-024014f1fb452f446' 2024-08-20T22:04:46.2128199Z Runner group name: 'Default' 2024-08-20T22:04:46.2129055Z Machine name: 'ip-10-0-44-197' 2024-08-20T22:04:46.2145537Z Testing runner upgrade compatibility 2024-08-20T22:04:46.3789978Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T22:04:46.3792275Z Actions: read 2024-08-20T22:04:46.3792865Z Attestations: read 2024-08-20T22:04:46.3793462Z Checks: read 2024-08-20T22:04:46.3793997Z Contents: read 2024-08-20T22:04:46.3794524Z Deployments: read 2024-08-20T22:04:46.3795120Z Discussions: read 2024-08-20T22:04:46.3795692Z Issues: read 2024-08-20T22:04:46.3796186Z Metadata: read 2024-08-20T22:04:46.3796753Z Packages: read 2024-08-20T22:04:46.3797249Z Pages: read 2024-08-20T22:04:46.3797747Z PullRequests: read 2024-08-20T22:04:46.3798396Z RepositoryProjects: read 2024-08-20T22:04:46.3799060Z SecurityEvents: read 2024-08-20T22:04:46.3799609Z Statuses: read 2024-08-20T22:04:46.3800419Z ##[endgroup] 2024-08-20T22:04:46.3803763Z Secret source: Actions 2024-08-20T22:04:46.3804639Z Prepare workflow directory 2024-08-20T22:04:46.5228214Z Prepare all required actions 2024-08-20T22:04:46.5400912Z Getting action download info 2024-08-20T22:04:46.7060207Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T22:04:47.2105750Z Download action repository 'pytorch/pytorch@main' (SHA:1ae5d5bb62141d7c9b1b0b66c66a462b4e10b1f2) 2024-08-20T22:04:51.2608229Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T22:04:51.4465189Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T22:04:51.6794557Z Getting action download info 2024-08-20T22:04:51.7962313Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T22:04:51.9756976Z Getting action download info 2024-08-20T22:04:52.0742833Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T22:04:52.2108475Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T22:04:52.2110635Z ##[group] Inputs 2024-08-20T22:04:52.2111106Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:04:52.2118444Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:04:52.2126571Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:04:52.2127768Z sync-tag: 2024-08-20T22:04:52.2128756Z timeout-minutes: 240 2024-08-20T22:04:52.2129096Z use-gha: 2024-08-20T22:04:52.2129385Z dashboard-tag: 2024-08-20T22:04:52.2129714Z s3-bucket: gha-artifacts 2024-08-20T22:04:52.2130067Z aws-role-to-assume: 2024-08-20T22:04:52.2130404Z ##[endgroup] 2024-08-20T22:04:52.2131492Z Complete job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:04:52.2798769Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T22:04:52.2939295Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T22:04:52.2951839Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:04:52.2952399Z ##[endgroup] 2024-08-20T22:04:54.5891172Z Runner Type: linux.g5.4xlarge.nvidia.gpu 2024-08-20T22:04:54.5891719Z Instance Type: g5.4xlarge 2024-08-20T22:04:54.5892448Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T22:04:54.5893007Z AMI ID: ami-06c68f701d8090592 2024-08-20T22:05:00.3884728Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T22:05:00.3885299Z with: 2024-08-20T22:05:00.3886217Z github-secret: *** 2024-08-20T22:05:00.3887204Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:05:00.3888223Z activate-with-label: false 2024-08-20T22:05:00.3888593Z label: with-ssh 2024-08-20T22:05:00.3888924Z remove-existing-keys: true 2024-08-20T22:05:00.3889292Z fail-silently: true 2024-08-20T22:05:00.3889612Z env: 2024-08-20T22:05:00.3889892Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:05:00.3890258Z ##[endgroup] 2024-08-20T22:05:00.4788359Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T22:05:00.4791014Z ciflow reference detected, attempting to extract PR number 2024-08-20T22:05:00.7526742Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T22:05:00.8485701Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T22:05:00.8486356Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T22:05:00.9049162Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T22:05:00.9062704Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T22:05:00.9086705Z Login using: ssh ec2-user@ec2-174-129-62-170.compute-1.amazonaws.com 2024-08-20T22:05:00.9087510Z All testing is done inside the container, to start an interactive session run: 2024-08-20T22:05:00.9088340Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:05:00.9221776Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T22:05:00.9222337Z with: 2024-08-20T22:05:00.9222612Z submodules: recursive 2024-08-20T22:05:00.9222943Z fetch-depth: 0 2024-08-20T22:05:00.9223231Z env: 2024-08-20T22:05:00.9223498Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:05:00.9223833Z ##[endgroup] 2024-08-20T22:05:00.9414528Z ##[group]Run retry () { 2024-08-20T22:05:00.9414924Z retry () { 2024-08-20T22:05:00.9415458Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T22:05:00.9416029Z } 2024-08-20T22:05:00.9416343Z echo "${GITHUB_WORKSPACE}" 2024-08-20T22:05:00.9416779Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T22:05:00.9417268Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:05:00.9417735Z else 2024-08-20T22:05:00.9418082Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:05:00.9418516Z fi 2024-08-20T22:05:00.9419092Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T22:05:00.9429123Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:05:00.9429647Z env: 2024-08-20T22:05:00.9429932Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:05:00.9430281Z NO_SUDO: 2024-08-20T22:05:00.9430564Z ##[endgroup] 2024-08-20T22:05:00.9462585Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:05:06.1453148Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T22:05:06.1453591Z with: 2024-08-20T22:05:06.1453932Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:05:06.1454386Z fetch-depth: 0 2024-08-20T22:05:06.1454715Z submodules: recursive 2024-08-20T22:05:06.1455056Z quiet-checkout: true 2024-08-20T22:05:06.1455413Z repository: pytorch/pytorch 2024-08-20T22:05:06.1455907Z token: *** 2024-08-20T22:05:06.1456205Z ssh-strict: true 2024-08-20T22:05:06.1456533Z persist-credentials: true 2024-08-20T22:05:06.1456894Z clean: true 2024-08-20T22:05:06.1457248Z sparse-checkout-cone-mode: true 2024-08-20T22:05:06.1457647Z lfs: false 2024-08-20T22:05:06.1457952Z set-safe-directory: true 2024-08-20T22:05:06.1458298Z env: 2024-08-20T22:05:06.1458579Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:05:06.1458924Z ##[endgroup] 2024-08-20T22:05:06.2449163Z Syncing repository: pytorch/pytorch 2024-08-20T22:05:06.2450741Z ##[group]Getting Git version info 2024-08-20T22:05:06.2451570Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:05:06.2452451Z [command]/usr/bin/git version 2024-08-20T22:05:06.2452831Z git version 2.40.1 2024-08-20T22:05:06.2458538Z ##[endgroup] 2024-08-20T22:05:06.2472508Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/96dca5b1-7743-4edb-8457-bb78935e7e15/.gitconfig' 2024-08-20T22:05:06.2479747Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/96dca5b1-7743-4edb-8457-bb78935e7e15' before making global git config changes 2024-08-20T22:05:06.2481058Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:05:06.2484317Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:05:06.2552459Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:05:06.2556647Z ##[group]Initializing the repository 2024-08-20T22:05:06.2559438Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:05:06.2682709Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T22:05:06.2683674Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T22:05:06.2684500Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T22:05:06.2685086Z hint: 2024-08-20T22:05:06.2685569Z hint: git config --global init.defaultBranch 2024-08-20T22:05:06.2686063Z hint: 2024-08-20T22:05:06.2687014Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T22:05:06.2688074Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T22:05:06.2688680Z hint: 2024-08-20T22:05:06.2689014Z hint: git branch -m 2024-08-20T22:05:06.2689780Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T22:05:06.2694664Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T22:05:06.2744811Z ##[endgroup] 2024-08-20T22:05:06.2745391Z ##[group]Disabling automatic garbage collection 2024-08-20T22:05:06.2747875Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T22:05:06.2789773Z ##[endgroup] 2024-08-20T22:05:06.2790302Z ##[group]Setting up auth 2024-08-20T22:05:06.2795142Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:05:06.2838988Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:05:06.3198949Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:05:06.3241201Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:05:06.3600263Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:05:06.3664654Z ##[endgroup] 2024-08-20T22:05:06.3665224Z ##[group]Fetching the repository 2024-08-20T22:05:06.3670794Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T22:05:09.2699198Z remote: Enumerating objects: 1008452 2024-08-20T22:05:09.2700570Z remote: Enumerating objects: 1010208, done. 2024-08-20T22:05:09.2702079Z remote: Counting objects: 0% (1/1756) 2024-08-20T22:05:09.2703529Z remote: Counting objects: 1% (18/1756) 2024-08-20T22:05:09.2704871Z remote: Counting objects: 2% (36/1756) 2024-08-20T22:05:09.2706193Z remote: Counting objects: 3% (53/1756) 2024-08-20T22:05:09.2707527Z remote: Counting objects: 4% (71/1756) 2024-08-20T22:05:09.2708555Z remote: Counting objects: 5% (88/1756) 2024-08-20T22:05:09.2709225Z remote: Counting objects: 6% (106/1756) 2024-08-20T22:05:09.2709817Z remote: Counting objects: 7% (123/1756) 2024-08-20T22:05:09.2710321Z remote: Counting objects: 8% (141/1756) 2024-08-20T22:05:09.2710913Z remote: Counting objects: 9% (159/1756) 2024-08-20T22:05:09.2711586Z remote: Counting objects: 10% (176/1756) 2024-08-20T22:05:09.2712254Z remote: Counting objects: 11% (194/1756) 2024-08-20T22:05:09.2712906Z remote: Counting objects: 12% (211/1756) 2024-08-20T22:05:09.2713510Z remote: Counting objects: 13% (229/1756) 2024-08-20T22:05:09.2714194Z remote: Counting objects: 14% (246/1756) 2024-08-20T22:05:09.2714862Z remote: Counting objects: 15% (264/1756) 2024-08-20T22:05:09.2715447Z remote: Counting objects: 16% (281/1756) 2024-08-20T22:05:09.2715991Z remote: Counting objects: 17% (299/1756) 2024-08-20T22:05:09.2716654Z remote: Counting objects: 18% (317/1756) 2024-08-20T22:05:09.2717297Z remote: Counting objects: 19% (334/1756) 2024-08-20T22:05:09.2717800Z remote: Counting objects: 20% (352/1756) 2024-08-20T22:05:09.2718355Z remote: Counting objects: 21% (369/1756) 2024-08-20T22:05:09.2719015Z remote: Counting objects: 22% (387/1756) 2024-08-20T22:05:09.2719683Z remote: Counting objects: 23% (404/1756) 2024-08-20T22:05:09.2720298Z remote: Counting objects: 24% (422/1756) 2024-08-20T22:05:09.2720818Z remote: Counting objects: 25% (439/1756) 2024-08-20T22:05:09.2722509Z remote: Counting objects: 26% (457/1756) 2024-08-20T22:05:09.2723211Z remote: Counting objects: 27% (475/1756) 2024-08-20T22:05:09.2723875Z remote: Counting objects: 28% (492/1756) 2024-08-20T22:05:09.2724494Z remote: Counting objects: 29% (510/1756) 2024-08-20T22:05:09.2724994Z remote: Counting objects: 30% (527/1756) 2024-08-20T22:05:09.2725589Z remote: Counting objects: 31% (545/1756) 2024-08-20T22:05:09.2726253Z remote: Counting objects: 32% (562/1756) 2024-08-20T22:05:09.2726816Z remote: Counting objects: 33% (580/1756) 2024-08-20T22:05:09.2727318Z remote: Counting objects: 34% (598/1756) 2024-08-20T22:05:09.2727808Z remote: Counting objects: 35% (615/1756) 2024-08-20T22:05:09.2728317Z remote: Counting objects: 36% (633/1756) 2024-08-20T22:05:09.2728817Z remote: Counting objects: 37% (650/1756) 2024-08-20T22:05:09.2729315Z remote: Counting objects: 38% (668/1756) 2024-08-20T22:05:09.2730034Z remote: Counting objects: 39% (685/1756) 2024-08-20T22:05:09.2730546Z remote: Counting objects: 40% (703/1756) 2024-08-20T22:05:09.2731046Z remote: Counting objects: 41% (720/1756) 2024-08-20T22:05:09.2731542Z remote: Counting objects: 42% (738/1756) 2024-08-20T22:05:09.2732032Z remote: Counting objects: 43% (756/1756) 2024-08-20T22:05:09.2732540Z remote: Counting objects: 44% (773/1756) 2024-08-20T22:05:09.2733040Z remote: Counting objects: 45% (791/1756) 2024-08-20T22:05:09.2733536Z remote: Counting objects: 46% (808/1756) 2024-08-20T22:05:09.2734026Z remote: Counting objects: 47% (826/1756) 2024-08-20T22:05:09.2734524Z remote: Counting objects: 48% (843/1756) 2024-08-20T22:05:09.2735021Z remote: Counting objects: 49% (861/1756) 2024-08-20T22:05:09.2735527Z remote: Counting objects: 50% (878/1756) 2024-08-20T22:05:09.2736025Z remote: Counting objects: 51% (896/1756) 2024-08-20T22:05:09.2736532Z remote: Counting objects: 52% (914/1756) 2024-08-20T22:05:09.2737030Z remote: Counting objects: 53% (931/1756) 2024-08-20T22:05:09.2737532Z remote: Counting objects: 54% (949/1756) 2024-08-20T22:05:09.2738024Z remote: Counting objects: 55% (966/1756) 2024-08-20T22:05:09.2738531Z remote: Counting objects: 56% (984/1756) 2024-08-20T22:05:09.2739045Z remote: Counting objects: 57% (1001/1756) 2024-08-20T22:05:09.2739565Z remote: Counting objects: 58% (1019/1756) 2024-08-20T22:05:09.2740094Z remote: Counting objects: 59% (1037/1756) 2024-08-20T22:05:09.2740618Z remote: Counting objects: 60% (1054/1756) 2024-08-20T22:05:09.2741136Z remote: Counting objects: 61% (1072/1756) 2024-08-20T22:05:09.2741759Z remote: Counting objects: 62% (1089/1756) 2024-08-20T22:05:09.2742301Z remote: Counting objects: 63% (1107/1756) 2024-08-20T22:05:09.2742823Z remote: Counting objects: 64% (1124/1756) 2024-08-20T22:05:09.2743359Z remote: Counting objects: 65% (1142/1756) 2024-08-20T22:05:09.2743881Z remote: Counting objects: 66% (1159/1756) 2024-08-20T22:05:09.2744395Z remote: Counting objects: 67% (1177/1756) 2024-08-20T22:05:09.2744917Z remote: Counting objects: 68% (1195/1756) 2024-08-20T22:05:09.2745442Z remote: Counting objects: 69% (1212/1756) 2024-08-20T22:05:09.2745959Z remote: Counting objects: 70% (1230/1756) 2024-08-20T22:05:09.2746469Z remote: Counting objects: 71% (1247/1756) 2024-08-20T22:05:09.2746990Z remote: Counting objects: 72% (1265/1756) 2024-08-20T22:05:09.2747988Z remote: Counting objects: 73% (1282/1756) 2024-08-20T22:05:09.2748507Z remote: Counting objects: 74% (1300/1756) 2024-08-20T22:05:09.2749017Z remote: Counting objects: 75% (1317/1756) 2024-08-20T22:05:09.2749539Z remote: Counting objects: 76% (1335/1756) 2024-08-20T22:05:09.2750213Z remote: Counting objects: 77% (1353/1756) 2024-08-20T22:05:09.2750744Z remote: Counting objects: 78% (1370/1756) 2024-08-20T22:05:09.2751257Z remote: Counting objects: 79% (1388/1756) 2024-08-20T22:05:09.2751776Z remote: Counting objects: 80% (1405/1756) 2024-08-20T22:05:09.2752306Z remote: Counting objects: 81% (1423/1756) 2024-08-20T22:05:09.2752826Z remote: Counting objects: 82% (1440/1756) 2024-08-20T22:05:09.2753336Z remote: Counting objects: 83% (1458/1756) 2024-08-20T22:05:09.2753859Z remote: Counting objects: 84% (1476/1756) 2024-08-20T22:05:09.2754376Z remote: Counting objects: 85% (1493/1756) 2024-08-20T22:05:09.2754895Z remote: Counting objects: 86% (1511/1756) 2024-08-20T22:05:09.2755408Z remote: Counting objects: 87% (1528/1756) 2024-08-20T22:05:09.2755927Z remote: Counting objects: 88% (1546/1756) 2024-08-20T22:05:09.2756444Z remote: Counting objects: 89% (1563/1756) 2024-08-20T22:05:09.2757094Z remote: Counting objects: 90% (1581/1756) 2024-08-20T22:05:09.2757607Z remote: Counting objects: 91% (1598/1756) 2024-08-20T22:05:09.2758128Z remote: Counting objects: 92% (1616/1756) 2024-08-20T22:05:09.2758697Z remote: Counting objects: 93% (1634/1756) 2024-08-20T22:05:09.2759217Z remote: Counting objects: 94% (1651/1756) 2024-08-20T22:05:09.2759729Z remote: Counting objects: 95% (1669/1756) 2024-08-20T22:05:09.2760249Z remote: Counting objects: 96% (1686/1756) 2024-08-20T22:05:09.2760767Z remote: Counting objects: 97% (1704/1756) 2024-08-20T22:05:09.2761282Z remote: Counting objects: 98% (1721/1756) 2024-08-20T22:05:09.2761793Z remote: Counting objects: 99% (1739/1756) 2024-08-20T22:05:09.2762311Z remote: Counting objects: 100% (1756/1756) 2024-08-20T22:05:09.2762868Z remote: Counting objects: 100% (1756/1756), done. 2024-08-20T22:05:09.2928531Z remote: Compressing objects: 0% (1/837) 2024-08-20T22:05:09.3204207Z remote: Compressing objects: 1% (9/837) 2024-08-20T22:05:09.3575501Z remote: Compressing objects: 2% (17/837) 2024-08-20T22:05:09.3971922Z remote: Compressing objects: 3% (26/837) 2024-08-20T22:05:09.4688656Z remote: Compressing objects: 4% (34/837) 2024-08-20T22:05:09.5108664Z remote: Compressing objects: 5% (42/837) 2024-08-20T22:05:09.5686965Z remote: Compressing objects: 6% (51/837) 2024-08-20T22:05:09.5992901Z remote: Compressing objects: 7% (59/837) 2024-08-20T22:05:09.6360502Z remote: Compressing objects: 8% (67/837) 2024-08-20T22:05:09.6562448Z remote: Compressing objects: 9% (76/837) 2024-08-20T22:05:09.6828021Z remote: Compressing objects: 10% (84/837) 2024-08-20T22:05:09.7024486Z remote: Compressing objects: 11% (93/837) 2024-08-20T22:05:09.7192312Z remote: Compressing objects: 12% (101/837) 2024-08-20T22:05:09.7350351Z remote: Compressing objects: 13% (109/837) 2024-08-20T22:05:09.7464985Z remote: Compressing objects: 14% (118/837) 2024-08-20T22:05:09.7549320Z remote: Compressing objects: 15% (126/837) 2024-08-20T22:05:09.7638026Z remote: Compressing objects: 16% (134/837) 2024-08-20T22:05:09.7691874Z remote: Compressing objects: 17% (143/837) 2024-08-20T22:05:09.7743902Z remote: Compressing objects: 18% (151/837) 2024-08-20T22:05:09.7782258Z remote: Compressing objects: 19% (160/837) 2024-08-20T22:05:09.7813599Z remote: Compressing objects: 20% (168/837) 2024-08-20T22:05:09.7837705Z remote: Compressing objects: 21% (176/837) 2024-08-20T22:05:09.7842822Z remote: Compressing objects: 22% (185/837) 2024-08-20T22:05:09.7847678Z remote: Compressing objects: 23% (193/837) 2024-08-20T22:05:09.7858569Z remote: Compressing objects: 24% (201/837) 2024-08-20T22:05:09.7864328Z remote: Compressing objects: 25% (210/837) 2024-08-20T22:05:09.7878250Z remote: Compressing objects: 26% (218/837) 2024-08-20T22:05:09.7886684Z remote: Compressing objects: 27% (226/837) 2024-08-20T22:05:09.7905272Z remote: Compressing objects: 28% (235/837) 2024-08-20T22:05:09.7909910Z remote: Compressing objects: 29% (243/837) 2024-08-20T22:05:09.7925114Z remote: Compressing objects: 30% (252/837) 2024-08-20T22:05:09.7933264Z remote: Compressing objects: 31% (260/837) 2024-08-20T22:05:09.7944151Z remote: Compressing objects: 32% (268/837) 2024-08-20T22:05:09.7949272Z remote: Compressing objects: 33% (277/837) 2024-08-20T22:05:09.7954261Z remote: Compressing objects: 34% (285/837) 2024-08-20T22:05:09.7962563Z remote: Compressing objects: 35% (293/837) 2024-08-20T22:05:09.7969376Z remote: Compressing objects: 36% (302/837) 2024-08-20T22:05:09.7979171Z remote: Compressing objects: 37% (310/837) 2024-08-20T22:05:09.7984276Z remote: Compressing objects: 38% (319/837) 2024-08-20T22:05:09.7990840Z remote: Compressing objects: 39% (327/837) 2024-08-20T22:05:09.7996698Z remote: Compressing objects: 40% (335/837) 2024-08-20T22:05:09.8003867Z remote: Compressing objects: 41% (344/837) 2024-08-20T22:05:09.8012183Z remote: Compressing objects: 42% (352/837) 2024-08-20T22:05:09.8018117Z remote: Compressing objects: 43% (360/837) 2024-08-20T22:05:09.8025254Z remote: Compressing objects: 44% (369/837) 2024-08-20T22:05:09.8031376Z remote: Compressing objects: 45% (377/837) 2024-08-20T22:05:09.8034916Z remote: Compressing objects: 46% (386/837) 2024-08-20T22:05:09.8039146Z remote: Compressing objects: 47% (394/837) 2024-08-20T22:05:09.8043117Z remote: Compressing objects: 48% (402/837) 2024-08-20T22:05:09.8046202Z remote: Compressing objects: 49% (411/837) 2024-08-20T22:05:09.8049950Z remote: Compressing objects: 50% (419/837) 2024-08-20T22:05:09.8054350Z remote: Compressing objects: 51% (427/837) 2024-08-20T22:05:09.8056231Z remote: Compressing objects: 52% (436/837) 2024-08-20T22:05:09.8058414Z remote: Compressing objects: 53% (444/837) 2024-08-20T22:05:09.8061183Z remote: Compressing objects: 54% (452/837) 2024-08-20T22:05:09.8062719Z remote: Compressing objects: 55% (461/837) 2024-08-20T22:05:09.8064894Z remote: Compressing objects: 56% (469/837) 2024-08-20T22:05:09.8066110Z remote: Compressing objects: 57% (478/837) 2024-08-20T22:05:09.8067327Z remote: Compressing objects: 58% (486/837) 2024-08-20T22:05:09.8068650Z remote: Compressing objects: 59% (494/837) 2024-08-20T22:05:09.8069881Z remote: Compressing objects: 60% (503/837) 2024-08-20T22:05:09.8070588Z remote: Compressing objects: 61% (511/837) 2024-08-20T22:05:09.8071460Z remote: Compressing objects: 62% (519/837) 2024-08-20T22:05:09.8072476Z remote: Compressing objects: 63% (528/837) 2024-08-20T22:05:09.8077271Z remote: Compressing objects: 64% (536/837) 2024-08-20T22:05:09.8083523Z remote: Compressing objects: 65% (545/837) 2024-08-20T22:05:09.8088906Z remote: Compressing objects: 66% (553/837) 2024-08-20T22:05:09.8093608Z remote: Compressing objects: 67% (561/837) 2024-08-20T22:05:09.8098036Z remote: Compressing objects: 68% (570/837) 2024-08-20T22:05:09.8102320Z remote: Compressing objects: 69% (578/837) 2024-08-20T22:05:09.8106691Z remote: Compressing objects: 70% (586/837) 2024-08-20T22:05:09.8110129Z remote: Compressing objects: 71% (595/837) 2024-08-20T22:05:09.8113927Z remote: Compressing objects: 72% (603/837) 2024-08-20T22:05:09.8119710Z remote: Compressing objects: 73% (612/837) 2024-08-20T22:05:09.8124030Z remote: Compressing objects: 74% (620/837) 2024-08-20T22:05:09.8126042Z remote: Compressing objects: 75% (628/837) 2024-08-20T22:05:09.8130457Z remote: Compressing objects: 76% (637/837) 2024-08-20T22:05:09.8135336Z remote: Compressing objects: 77% (645/837) 2024-08-20T22:05:09.8136087Z remote: Compressing objects: 78% (653/837) 2024-08-20T22:05:09.8138634Z remote: Compressing objects: 79% (662/837) 2024-08-20T22:05:09.8142024Z remote: Compressing objects: 80% (670/837) 2024-08-20T22:05:09.8143950Z remote: Compressing objects: 81% (678/837) 2024-08-20T22:05:09.8147510Z remote: Compressing objects: 82% (687/837) 2024-08-20T22:05:09.8149448Z remote: Compressing objects: 83% (695/837) 2024-08-20T22:05:09.8151533Z remote: Compressing objects: 84% (704/837) 2024-08-20T22:05:09.8154425Z remote: Compressing objects: 85% (712/837) 2024-08-20T22:05:09.8155850Z remote: Compressing objects: 86% (720/837) 2024-08-20T22:05:09.8157938Z remote: Compressing objects: 87% (729/837) 2024-08-20T22:05:09.8161014Z remote: Compressing objects: 88% (737/837) 2024-08-20T22:05:09.8164722Z remote: Compressing objects: 89% (745/837) 2024-08-20T22:05:09.8166663Z remote: Compressing objects: 90% (754/837) 2024-08-20T22:05:09.8168792Z remote: Compressing objects: 91% (762/837) 2024-08-20T22:05:09.8171284Z remote: Compressing objects: 92% (771/837) 2024-08-20T22:05:09.8172526Z remote: Compressing objects: 93% (779/837) 2024-08-20T22:05:09.8173245Z remote: Compressing objects: 94% (787/837) 2024-08-20T22:05:09.8174233Z remote: Compressing objects: 95% (796/837) 2024-08-20T22:05:09.8176326Z remote: Compressing objects: 96% (804/837) 2024-08-20T22:05:09.8177941Z remote: Compressing objects: 97% (812/837) 2024-08-20T22:05:09.8180002Z remote: Compressing objects: 98% (821/837) 2024-08-20T22:05:09.8181497Z remote: Compressing objects: 99% (829/837) 2024-08-20T22:05:09.8182901Z remote: Compressing objects: 100% (837/837) 2024-08-20T22:05:09.8183674Z remote: Compressing objects: 100% (837/837), done. 2024-08-20T22:05:33.0343055Z remote: Total 1010208 (delta 1154), reused 1394 (delta 916), pack-reused 1008452 (from 1) 2024-08-20T22:05:54.6803971Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T22:05:54.6840756Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:05:54.6845952Z ##[endgroup] 2024-08-20T22:05:54.6849865Z ##[group]Determining the checkout info 2024-08-20T22:05:54.6850524Z ##[endgroup] 2024-08-20T22:05:54.6851221Z ##[group]Checking out the ref 2024-08-20T22:05:54.6852167Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:05:56.4258767Z ##[endgroup] 2024-08-20T22:05:56.4259394Z ##[group]Setting up auth for fetching submodules 2024-08-20T22:05:56.4263883Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:05:56.4324931Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T22:05:56.4366393Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T22:05:56.4407494Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T22:05:56.4446153Z ##[endgroup] 2024-08-20T22:05:56.4446703Z ##[group]Fetching submodules 2024-08-20T22:05:56.4450330Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T22:05:56.4834197Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T22:05:56.5205542Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T22:05:56.5207183Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T22:05:56.5210796Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T22:05:56.5214684Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T22:05:56.5218300Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T22:05:56.5221757Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T22:05:56.5225561Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T22:05:56.5229443Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T22:05:56.5233232Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T22:05:56.5237343Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T22:05:56.5241344Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T22:05:56.5245448Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T22:05:56.5250301Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T22:05:56.5254443Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T22:05:56.5258580Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T22:05:56.5264406Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:05:56.5272326Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T22:05:56.5276936Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T22:05:56.5281378Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T22:05:56.5286231Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T22:05:56.5290981Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T22:05:56.5295848Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T22:05:56.5300682Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T22:05:56.5305945Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T22:05:56.5310811Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T22:05:56.5316221Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T22:05:56.5321156Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T22:05:56.5326411Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T22:05:56.5331756Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T22:05:56.5337195Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T22:05:56.5342852Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T22:05:56.5348909Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T22:05:56.5354336Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T22:05:56.5363278Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T22:05:56.5398552Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T22:05:56.8209075Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T22:05:57.0046260Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T22:05:57.1720538Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T22:05:57.4105404Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T22:05:59.3545098Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T22:06:11.0031635Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T22:06:11.4079863Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T22:06:11.8630604Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T22:06:12.5142122Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T22:06:13.7769249Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T22:06:15.8540289Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T22:06:21.7139035Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T22:06:23.1938929Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T22:06:24.9801727Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T22:06:26.3146229Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T22:06:26.7522873Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T22:06:27.0794381Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T22:06:28.3093289Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T22:06:28.6990554Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T22:06:28.9524276Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T22:06:30.5830636Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T22:06:31.2859590Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T22:06:31.9535732Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T22:06:38.1350657Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T22:06:40.1103228Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T22:06:44.3947364Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T22:06:44.6220765Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T22:06:54.2962215Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T22:06:54.4710043Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T22:06:54.6631419Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T22:06:55.5863036Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T22:06:55.8691851Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T22:06:56.4996065Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T22:06:56.9236793Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T22:06:56.9387267Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T22:06:56.9507455Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T22:06:56.9816356Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T22:06:57.0261816Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T22:06:58.2079290Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T22:06:58.2368420Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T22:06:58.2903581Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T22:06:58.4042155Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T22:06:58.4449196Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T22:06:59.0730454Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T22:06:59.3648760Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T22:06:59.4589876Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T22:06:59.4612204Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:06:59.4614346Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:06:59.4618110Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:06:59.4621648Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:06:59.4625375Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:06:59.4656991Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T22:07:00.4400234Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T22:07:01.0497030Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T22:07:03.2535446Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T22:07:04.2849767Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T22:07:04.6270321Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T22:07:04.7388002Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T22:07:05.2484911Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T22:07:05.3206286Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T22:07:05.3360271Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T22:07:05.4897622Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T22:07:05.5360143Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T22:07:05.5823062Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T22:07:05.6148321Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T22:07:05.6678206Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:07:05.6838917Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T22:07:05.6860993Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T22:07:05.6892520Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T22:07:18.8829823Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T22:07:18.9049430Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T22:07:19.0054597Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T22:07:19.0075164Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:07:19.0078406Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:07:19.0082095Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:07:19.0114812Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T22:07:19.5173750Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T22:07:20.7419303Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T22:07:21.9249965Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T22:07:21.9271699Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:07:21.9275113Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:07:21.9278882Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:07:21.9282540Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:07:21.9286887Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:07:21.9290828Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:07:21.9294840Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:07:21.9298945Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:07:21.9330915Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T22:07:22.7798900Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T22:07:23.1287228Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T22:07:24.2993832Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T22:07:24.5923427Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T22:07:25.1072261Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T22:07:26.1296069Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T22:07:33.3053634Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T22:07:33.7127912Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T22:07:33.7370607Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T22:07:33.7807819Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T22:07:33.7978743Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T22:07:33.7998000Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:07:33.8031109Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T22:07:34.0802181Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T22:07:34.1031888Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T22:07:34.1517133Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T22:07:34.2717091Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T22:07:34.2921262Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T22:07:34.3380200Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T22:07:34.4044132Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T22:07:34.4507551Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T22:07:34.4802496Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T22:07:34.6071598Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T22:07:35.1229622Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T22:07:35.1265393Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T22:07:35.1268676Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T22:07:35.1301491Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T22:07:35.5873106Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T22:07:36.4754346Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T22:07:36.5177015Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T22:07:36.6069310Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T22:07:36.6091100Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:07:36.6094472Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:07:36.6098472Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:07:36.6102232Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:07:36.6106334Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:07:36.6110076Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:07:36.6113721Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:07:36.6117377Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:07:36.6155125Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T22:07:37.0056444Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T22:07:38.0306770Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T22:07:38.3265538Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T22:07:44.4479439Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T22:07:44.7167095Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T22:07:44.9422821Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T22:07:45.2311964Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T22:07:51.9088887Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T22:07:51.9567449Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T22:07:51.9757063Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T22:07:52.0972837Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T22:07:52.1137135Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T22:07:52.1319052Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T22:07:52.1522489Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T22:07:52.1541530Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:07:52.1545130Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:07:52.1580478Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T22:07:54.0998708Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T22:07:55.4649182Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T22:07:55.5190075Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:07:56.1690026Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T22:07:56.1836952Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T22:07:56.5039898Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T22:07:56.5063017Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:07:56.5066445Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T22:07:56.5099128Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T22:07:56.9243755Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T22:07:57.9989564Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T22:07:58.0798074Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T22:07:58.0921401Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T22:07:58.1079847Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T22:07:58.1533761Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T22:07:58.1881640Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T22:07:58.2372971Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T22:07:58.2709656Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T22:07:58.2730555Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:07:58.2734241Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:07:58.2737512Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:07:58.2740932Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:07:58.2774169Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T22:07:59.3605339Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T22:07:59.5779218Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T22:08:00.7740325Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T22:08:01.8663718Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T22:08:01.8863549Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T22:08:01.9657135Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T22:08:02.0008506Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T22:08:02.0031213Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:02.0063317Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T22:08:02.2054067Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T22:08:02.2106714Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T22:08:02.2471742Z Entering 'android/libs/fbjni' 2024-08-20T22:08:02.2523918Z Entering 'third_party/FP16' 2024-08-20T22:08:02.2576645Z Entering 'third_party/FXdiv' 2024-08-20T22:08:02.2629343Z Entering 'third_party/NNPACK' 2024-08-20T22:08:02.2682101Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:02.2733615Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:02.2802806Z Entering 'third_party/benchmark' 2024-08-20T22:08:02.2854550Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:02.2904825Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:02.2956378Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:02.3006986Z Entering 'third_party/cutlass' 2024-08-20T22:08:02.3067669Z Entering 'third_party/eigen' 2024-08-20T22:08:02.3122119Z Entering 'third_party/fbgemm' 2024-08-20T22:08:02.3174109Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:02.3229915Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:02.3281824Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:02.3339295Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:02.3393358Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:02.3447355Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:02.3504640Z Entering 'third_party/fmt' 2024-08-20T22:08:02.3555020Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:02.3605770Z Entering 'third_party/gloo' 2024-08-20T22:08:02.3656724Z Entering 'third_party/googletest' 2024-08-20T22:08:02.3707089Z Entering 'third_party/ideep' 2024-08-20T22:08:02.3756308Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:02.3815829Z Entering 'third_party/ittapi' 2024-08-20T22:08:02.3867728Z Entering 'third_party/kineto' 2024-08-20T22:08:02.3917952Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:02.3969983Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:02.4023627Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:02.4074556Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:02.4124910Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:02.4175063Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:02.4235529Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:02.4287900Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:02.4339533Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:02.4392568Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:02.4448390Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:02.4498980Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:02.4553173Z Entering 'third_party/mimalloc' 2024-08-20T22:08:02.4604277Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:02.4655814Z Entering 'third_party/nlohmann' 2024-08-20T22:08:02.4707179Z Entering 'third_party/onnx' 2024-08-20T22:08:02.4771560Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:02.4823228Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:02.4880578Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:02.4932297Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:02.4984870Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:02.5035320Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:02.5087094Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:02.5138600Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:02.5189920Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:02.5239797Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:02.5293416Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:02.5345412Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:02.5399519Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:02.5477988Z Entering 'third_party/pocketfft' 2024-08-20T22:08:02.5527669Z Entering 'third_party/protobuf' 2024-08-20T22:08:02.5580412Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:02.5631383Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:02.5686549Z Entering 'third_party/psimd' 2024-08-20T22:08:02.5737058Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:02.5787880Z Entering 'third_party/pybind11' 2024-08-20T22:08:02.5838387Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:02.5889345Z Entering 'third_party/sleef' 2024-08-20T22:08:02.5939991Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:02.5989865Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:02.6040031Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:02.6093876Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:02.6145257Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:02.6194587Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:02.6272093Z ##[endgroup] 2024-08-20T22:08:02.6275581Z ##[group]Persisting credentials for submodules 2024-08-20T22:08:02.6277409Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T22:08:02.6642964Z Entering 'android/libs/fbjni' 2024-08-20T22:08:02.6711540Z Entering 'third_party/FP16' 2024-08-20T22:08:02.6778948Z Entering 'third_party/FXdiv' 2024-08-20T22:08:02.6846828Z Entering 'third_party/NNPACK' 2024-08-20T22:08:02.6915492Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:02.6988377Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:02.7071418Z Entering 'third_party/benchmark' 2024-08-20T22:08:02.7138599Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:02.7206314Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:02.7274616Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:02.7343172Z Entering 'third_party/cutlass' 2024-08-20T22:08:02.7420463Z Entering 'third_party/eigen' 2024-08-20T22:08:02.7493667Z Entering 'third_party/fbgemm' 2024-08-20T22:08:02.7561019Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:02.7628145Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:02.7695969Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:02.7768995Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:02.7835546Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:02.7904950Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:02.7977585Z Entering 'third_party/fmt' 2024-08-20T22:08:02.8049689Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:02.8117823Z Entering 'third_party/gloo' 2024-08-20T22:08:02.8185994Z Entering 'third_party/googletest' 2024-08-20T22:08:02.8253835Z Entering 'third_party/ideep' 2024-08-20T22:08:02.8319547Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:02.8396179Z Entering 'third_party/ittapi' 2024-08-20T22:08:02.8463663Z Entering 'third_party/kineto' 2024-08-20T22:08:02.8529052Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:02.8596004Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:02.8674296Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:02.8741649Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:02.8809349Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:02.8875258Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:02.8947564Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:02.9020149Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:02.9092143Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:02.9160812Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:02.9231135Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:02.9298197Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:02.9367713Z Entering 'third_party/mimalloc' 2024-08-20T22:08:02.9436518Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:02.9505541Z Entering 'third_party/nlohmann' 2024-08-20T22:08:02.9578785Z Entering 'third_party/onnx' 2024-08-20T22:08:02.9658755Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:02.9728536Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:02.9801411Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:02.9870579Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:02.9937927Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:03.0006010Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:03.0073129Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:03.0141685Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:03.0209723Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:03.0277003Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:03.0342719Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:03.0412911Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:03.0483646Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:03.0573297Z Entering 'third_party/pocketfft' 2024-08-20T22:08:03.0640860Z Entering 'third_party/protobuf' 2024-08-20T22:08:03.0711652Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:03.0778579Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:03.0851024Z Entering 'third_party/psimd' 2024-08-20T22:08:03.0920197Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:03.0989318Z Entering 'third_party/pybind11' 2024-08-20T22:08:03.1056658Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:03.1129072Z Entering 'third_party/sleef' 2024-08-20T22:08:03.1200202Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:03.1267177Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:03.1334164Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:03.1406087Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:03.1474831Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:03.1538995Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:03.1631707Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T22:08:03.1997978Z Entering 'android/libs/fbjni' 2024-08-20T22:08:03.2059517Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T22:08:03.2082285Z Entering 'third_party/FP16' 2024-08-20T22:08:03.2144869Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T22:08:03.2166546Z Entering 'third_party/FXdiv' 2024-08-20T22:08:03.2227657Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T22:08:03.2249048Z Entering 'third_party/NNPACK' 2024-08-20T22:08:03.2310597Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T22:08:03.2332932Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:03.2396117Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T22:08:03.2418277Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:03.2482597Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T22:08:03.2529846Z Entering 'third_party/benchmark' 2024-08-20T22:08:03.2584613Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:03.2605939Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:03.2671909Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T22:08:03.2693395Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:03.2755009Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:08:03.2777655Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:03.2839497Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T22:08:03.2861779Z Entering 'third_party/cutlass' 2024-08-20T22:08:03.2923216Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:08:03.2953058Z Entering 'third_party/eigen' 2024-08-20T22:08:03.3013084Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T22:08:03.3037031Z Entering 'third_party/fbgemm' 2024-08-20T22:08:03.3100043Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T22:08:03.3120344Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:03.3183166Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T22:08:03.3203640Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:03.3268373Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:08:03.3289540Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:03.3352521Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:08:03.3379902Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:03.3442252Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.3463975Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:03.3525002Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T22:08:03.3550710Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:03.3616714Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T22:08:03.3641028Z Entering 'third_party/fmt' 2024-08-20T22:08:03.3701936Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:08:03.3723881Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:03.3786268Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T22:08:03.3808546Z Entering 'third_party/gloo' 2024-08-20T22:08:03.3870107Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T22:08:03.3892498Z Entering 'third_party/googletest' 2024-08-20T22:08:03.3956187Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.3978482Z Entering 'third_party/ideep' 2024-08-20T22:08:03.4039678Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T22:08:03.4061031Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:03.4122559Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T22:08:03.4153117Z Entering 'third_party/ittapi' 2024-08-20T22:08:03.4213413Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T22:08:03.4234849Z Entering 'third_party/kineto' 2024-08-20T22:08:03.4298288Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T22:08:03.4318266Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:03.4382447Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T22:08:03.4401998Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:03.4464202Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T22:08:03.4486510Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:03.4550283Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T22:08:03.4572795Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:03.4635773Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:08:03.4656537Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:03.4720056Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T22:08:03.4738612Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:03.4802791Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T22:08:03.4826508Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:03.4891821Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T22:08:03.4913276Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:03.4976582Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.4999142Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:03.5062101Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T22:08:03.5084230Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:03.5146876Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T22:08:03.5174162Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:03.5235256Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T22:08:03.5256329Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:03.5318908Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.5341590Z Entering 'third_party/mimalloc' 2024-08-20T22:08:03.5404423Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T22:08:03.5426952Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:03.5495338Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T22:08:03.5518297Z Entering 'third_party/nlohmann' 2024-08-20T22:08:03.5580030Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T22:08:03.5603651Z Entering 'third_party/onnx' 2024-08-20T22:08:03.5667758Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T22:08:03.5702926Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:03.5766557Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:03.5788463Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:03.5853016Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:08:03.5879454Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:03.5942498Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T22:08:03.5964831Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:03.6025969Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:03.6045962Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:03.6107663Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.6127877Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:03.6188943Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T22:08:03.6209255Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:03.6270899Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T22:08:03.6292859Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:03.6355124Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T22:08:03.6375906Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:03.6437275Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T22:08:03.6459531Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:03.6520793Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T22:08:03.6539305Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:03.6602161Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T22:08:03.6624953Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:03.6687502Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T22:08:03.6710909Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:03.6772681Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T22:08:03.6815600Z Entering 'third_party/pocketfft' 2024-08-20T22:08:03.6877838Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T22:08:03.6899849Z Entering 'third_party/protobuf' 2024-08-20T22:08:03.6962313Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T22:08:03.6985326Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:03.7046273Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:03.7067427Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:03.7127975Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.7153006Z Entering 'third_party/psimd' 2024-08-20T22:08:03.7220125Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T22:08:03.7241597Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:03.7305264Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T22:08:03.7326817Z Entering 'third_party/pybind11' 2024-08-20T22:08:03.7392913Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:08:03.7416069Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:03.7482020Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T22:08:03.7504797Z Entering 'third_party/sleef' 2024-08-20T22:08:03.7569296Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T22:08:03.7595500Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:03.7658133Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T22:08:03.7678973Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:03.7741899Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.7766230Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:03.7829761Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T22:08:03.7853535Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:03.7916464Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T22:08:03.7938276Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:03.8001172Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:08:03.8019582Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:03.8082823Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T22:08:03.8847871Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T22:08:03.9219824Z Entering 'android/libs/fbjni' 2024-08-20T22:08:03.9275262Z Entering 'third_party/FP16' 2024-08-20T22:08:03.9329106Z Entering 'third_party/FXdiv' 2024-08-20T22:08:03.9381292Z Entering 'third_party/NNPACK' 2024-08-20T22:08:03.9434509Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:03.9488749Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:03.9557343Z Entering 'third_party/benchmark' 2024-08-20T22:08:03.9609471Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:03.9660327Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:03.9712451Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:03.9764633Z Entering 'third_party/cutlass' 2024-08-20T22:08:03.9822836Z Entering 'third_party/eigen' 2024-08-20T22:08:03.9879251Z Entering 'third_party/fbgemm' 2024-08-20T22:08:03.9930046Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:03.9982809Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:04.0034353Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:04.0094907Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:04.0151178Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:04.0204764Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:04.0261223Z Entering 'third_party/fmt' 2024-08-20T22:08:04.0315341Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:04.0368081Z Entering 'third_party/gloo' 2024-08-20T22:08:04.0421137Z Entering 'third_party/googletest' 2024-08-20T22:08:04.0478961Z Entering 'third_party/ideep' 2024-08-20T22:08:04.0528038Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:04.0589568Z Entering 'third_party/ittapi' 2024-08-20T22:08:04.0640364Z Entering 'third_party/kineto' 2024-08-20T22:08:04.0691355Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:04.0741752Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:04.0795061Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:04.0849025Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:04.0899853Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:04.0946988Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:04.1003732Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:04.1054206Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:04.1104851Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:04.1156512Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:04.1210899Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:04.1261905Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:04.1316117Z Entering 'third_party/mimalloc' 2024-08-20T22:08:04.1371973Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:04.1424326Z Entering 'third_party/nlohmann' 2024-08-20T22:08:04.1478198Z Entering 'third_party/onnx' 2024-08-20T22:08:04.1543077Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:04.1595337Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:04.1657945Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:04.1710008Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:04.1761382Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:04.1811780Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:04.1864295Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:04.1917873Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:04.1971636Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:04.2023425Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:04.2074902Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:04.2132841Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:04.2187365Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:04.2262146Z Entering 'third_party/pocketfft' 2024-08-20T22:08:04.2319528Z Entering 'third_party/protobuf' 2024-08-20T22:08:04.2374314Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:04.2424343Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:04.2481988Z Entering 'third_party/psimd' 2024-08-20T22:08:04.2536716Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:04.2589997Z Entering 'third_party/pybind11' 2024-08-20T22:08:04.2641664Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:04.2694805Z Entering 'third_party/sleef' 2024-08-20T22:08:04.2748216Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:04.2800719Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:04.2850080Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:04.2901093Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:04.2952173Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:04.3001142Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:04.3078883Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T22:08:04.3447462Z Entering 'android/libs/fbjni' 2024-08-20T22:08:04.3500317Z Entering 'third_party/FP16' 2024-08-20T22:08:04.3554047Z Entering 'third_party/FXdiv' 2024-08-20T22:08:04.3606036Z Entering 'third_party/NNPACK' 2024-08-20T22:08:04.3658821Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:04.3711051Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:04.3780597Z Entering 'third_party/benchmark' 2024-08-20T22:08:04.3833805Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:04.3888299Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:04.3942410Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:04.3996255Z Entering 'third_party/cutlass' 2024-08-20T22:08:04.4057924Z Entering 'third_party/eigen' 2024-08-20T22:08:04.4111470Z Entering 'third_party/fbgemm' 2024-08-20T22:08:04.4162821Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:04.4213557Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:04.4265180Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:04.4322210Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:04.4375029Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:04.4433487Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:04.4491659Z Entering 'third_party/fmt' 2024-08-20T22:08:04.4544858Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:04.4599518Z Entering 'third_party/gloo' 2024-08-20T22:08:04.4651833Z Entering 'third_party/googletest' 2024-08-20T22:08:04.4705187Z Entering 'third_party/ideep' 2024-08-20T22:08:04.4756662Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:04.4816624Z Entering 'third_party/ittapi' 2024-08-20T22:08:04.4870307Z Entering 'third_party/kineto' 2024-08-20T22:08:04.4922431Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:04.4975797Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:04.5029681Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:04.5081846Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:04.5133451Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:04.5183959Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:04.5243815Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:04.5295691Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:04.5346215Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:04.5397951Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:04.5452107Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:04.5503469Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:04.5559078Z Entering 'third_party/mimalloc' 2024-08-20T22:08:04.5611663Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:04.5666871Z Entering 'third_party/nlohmann' 2024-08-20T22:08:04.5719666Z Entering 'third_party/onnx' 2024-08-20T22:08:04.5785342Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:04.5836893Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:04.5894915Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:04.5947591Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:04.6002280Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:04.6054301Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:04.6105445Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:04.6157788Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:04.6208296Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:04.6261250Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:04.6309068Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:04.6363273Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:04.6419183Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:04.6489888Z Entering 'third_party/pocketfft' 2024-08-20T22:08:04.6543332Z Entering 'third_party/protobuf' 2024-08-20T22:08:04.6598283Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:04.6648552Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:04.6710354Z Entering 'third_party/psimd' 2024-08-20T22:08:04.6762953Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:04.6814296Z Entering 'third_party/pybind11' 2024-08-20T22:08:04.6866918Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:04.6919845Z Entering 'third_party/sleef' 2024-08-20T22:08:04.6974693Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:04.7026998Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:04.7079668Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:04.7134063Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:04.7186044Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:04.7235571Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:04.7310942Z ##[endgroup] 2024-08-20T22:08:04.7361933Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T22:08:04.7399485Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T22:08:04.7614613Z Prepare all required actions 2024-08-20T22:08:04.7615067Z Getting action download info 2024-08-20T22:08:04.8900003Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T22:08:04.8900433Z env: 2024-08-20T22:08:04.8900705Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.8901043Z ##[endgroup] 2024-08-20T22:08:04.8964859Z ##[group]Run set -euo pipefail 2024-08-20T22:08:04.8965314Z set -euo pipefail 2024-08-20T22:08:04.8965723Z function get_ec2_metadata() { 2024-08-20T22:08:04.8966277Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T22:08:04.8967243Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T22:08:04.8968034Z  category=$1 2024-08-20T22:08:04.8968586Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T22:08:04.8969249Z  runner_name_str=i-024014f1fb452f446 2024-08-20T22:08:04.8969757Z  if [[ -f /.inarc ]]; then 2024-08-20T22:08:04.8970278Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T22:08:04.8970862Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T22:08:04.8971591Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T22:08:04.8972218Z  else 2024-08-20T22:08:04.8972720Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T22:08:04.8973309Z  fi 2024-08-20T22:08:04.8973603Z } 2024-08-20T22:08:04.8973968Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T22:08:04.8974569Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T22:08:04.8975246Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T22:08:04.8975837Z echo "system info $(uname -a)" 2024-08-20T22:08:04.8984966Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:04.8985482Z env: 2024-08-20T22:08:04.8985775Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.8986138Z ##[endgroup] 2024-08-20T22:08:04.9084914Z ami-id: ami-06c68f701d8090592 2024-08-20T22:08:04.9142967Z instance-id: i-024014f1fb452f446 2024-08-20T22:08:04.9197332Z instance-type: g5.4xlarge 2024-08-20T22:08:04.9212207Z system info Linux ip-10-0-44-197.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T22:08:04.9244081Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:08:04.9245016Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:08:04.9254003Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:04.9254513Z env: 2024-08-20T22:08:04.9254804Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.9255164Z ##[endgroup] 2024-08-20T22:08:04.9333457Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T22:08:04.9334073Z if systemctl is-active --quiet docker; then 2024-08-20T22:08:04.9334808Z  echo "Docker daemon is running..."; 2024-08-20T22:08:04.9335447Z else 2024-08-20T22:08:04.9335946Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T22:08:04.9337079Z fi 2024-08-20T22:08:04.9345843Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:04.9346347Z env: 2024-08-20T22:08:04.9346770Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.9347951Z ##[endgroup] 2024-08-20T22:08:04.9435664Z Docker daemon is running... 2024-08-20T22:08:04.9486143Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:08:04.9486717Z with: 2024-08-20T22:08:04.9486994Z shell: bash 2024-08-20T22:08:04.9487287Z timeout_minutes: 5 2024-08-20T22:08:04.9487617Z max_attempts: 3 2024-08-20T22:08:04.9487936Z retry_wait_seconds: 30 2024-08-20T22:08:04.9491172Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T22:08:04.9494275Z polling_interval_seconds: 1 2024-08-20T22:08:04.9494651Z warning_on_retry: true 2024-08-20T22:08:04.9495011Z continue_on_error: false 2024-08-20T22:08:04.9495355Z env: 2024-08-20T22:08:04.9495629Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.9495985Z AWS_RETRY_MODE: standard 2024-08-20T22:08:04.9496339Z AWS_MAX_ATTEMPTS: 5 2024-08-20T22:08:04.9496687Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T22:08:04.9497073Z ##[endgroup] 2024-08-20T22:08:06.2742245Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:08:06.2743291Z Configure a credential helper to remove this warning. See 2024-08-20T22:08:06.2744188Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:08:06.2744720Z 2024-08-20T22:08:06.2744841Z Login Succeeded 2024-08-20T22:08:07.0063832Z Command completed after 1 attempt(s). 2024-08-20T22:08:07.0125875Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:08:07.0126607Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:08:07.0127276Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:08:07.0136253Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:07.0136755Z env: 2024-08-20T22:08:07.0137034Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:07.0137381Z ##[endgroup] 2024-08-20T22:08:07.0229325Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:08:07.0230102Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:08:07.0230717Z # shellcheck disable=SC2046 2024-08-20T22:08:07.0231180Z docker stop $(docker ps -q) || true 2024-08-20T22:08:07.0231657Z # Prune all of the docker images 2024-08-20T22:08:07.0232115Z docker system prune -af 2024-08-20T22:08:07.0240500Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:07.0241005Z env: 2024-08-20T22:08:07.0241281Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:07.0241631Z ##[endgroup] 2024-08-20T22:08:07.0563918Z "docker stop" requires at least 1 argument. 2024-08-20T22:08:07.0564666Z See 'docker stop --help'. 2024-08-20T22:08:07.0564925Z 2024-08-20T22:08:07.0565153Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T22:08:07.0565536Z 2024-08-20T22:08:07.0565697Z Stop one or more running containers 2024-08-20T22:08:09.9398858Z Deleted Images: 2024-08-20T22:08:09.9399548Z untagged: pytorch/manylinux-builder:cuda12.1 2024-08-20T22:08:09.9400500Z untagged: pytorch/manylinux-builder@sha256:1ceaa79868583367287b01793bb482334e53f31ed3974a78cd271b02247df451 2024-08-20T22:08:09.9401796Z deleted: sha256:f4fc3a9f2fd055e5b72e4feda5bbbd8c6e79ef22b6071d45945eb8ec64042f41 2024-08-20T22:08:09.9403068Z deleted: sha256:70032a09fef4ce88f555f69ad94ca1943d316b6f3b82da8dc7c2821690e88eed 2024-08-20T22:08:09.9404011Z deleted: sha256:a56cfbfa766fde9a9ee399928a535662d561613c3e54f9bf7d42a22557e05941 2024-08-20T22:08:09.9404942Z deleted: sha256:26c2625d88a3265ee12a5179363793d84e44778efa3b0565f571020d370121a9 2024-08-20T22:08:09.9405849Z deleted: sha256:00f9eb5fad579947541dd876721cd7e111bcbd38ca057b4c9c7c5d5130b45561 2024-08-20T22:08:09.9406780Z deleted: sha256:4bce2aee5079b7a763b5022d7617acfa9dc05c721b799345424836fff1bec24f 2024-08-20T22:08:09.9407713Z deleted: sha256:f572e0d9d5a866e08e3e512558dc935b53506c01916dcf5eb41c562cafca97c4 2024-08-20T22:08:09.9408839Z deleted: sha256:276ca07dabc1b6ec9a5577d3fe2fa1b0912f2c14eb1135fbfd5e883dadf96a55 2024-08-20T22:08:09.9409904Z deleted: sha256:413d9646e2cdd6bb7d142b7e2b44e80b8e55b0ea3101a06d2ae462bd98d41b53 2024-08-20T22:08:09.9410921Z deleted: sha256:eb7e8fa961c4c9ce0d409c41e2db44a5abf11078c04dcd727065808670502776 2024-08-20T22:08:09.9411953Z deleted: sha256:f052aaceea687692e6797adac86f83463ca13c7acb13399f3fe60671d914d837 2024-08-20T22:08:09.9413003Z deleted: sha256:95abcbbd6fe3402acb416254069895c430be984f650e0beb61e0b96dc9d4e3a1 2024-08-20T22:08:09.9413867Z deleted: sha256:1f1575a0dc7eafbe0bc24349a498188dc66b0c7afe0a92ad3177982dc53d1d7f 2024-08-20T22:08:09.9414734Z deleted: sha256:e8516cf5f8eaea7208825a0e74bc6ccc2425b2eeb87bf835fba068f096435065 2024-08-20T22:08:09.9415576Z deleted: sha256:50e753ec0ac354845824b7647dcb5c80fc1ecdf20246a62ef10afe70d2cba914 2024-08-20T22:08:09.9416420Z deleted: sha256:9a3a953a80dac55ead243068e4801c591294ce2467a7f2ab37acf37e7b50623d 2024-08-20T22:08:09.9417355Z deleted: sha256:8b589e158e3986bebcf7bfa8daf21e07ea295d0129753a23b59dc6016212d07a 2024-08-20T22:08:09.9418177Z deleted: sha256:9780567f56682e7ccb6519d6a582a4233a94a473e8319fdce963a938e8348437 2024-08-20T22:08:09.9418982Z deleted: sha256:83f38e31da6d72b6c7a862161962449d865590ecee2da9f77ce4a146acc998b0 2024-08-20T22:08:09.9419796Z deleted: sha256:b23603a0fe1921a9b378cd4f49db9da250b4f8026d73c027f355725059ea9ea8 2024-08-20T22:08:09.9420604Z deleted: sha256:6fc31f41ca816c5630174d83c904113d618ef54ee97610240e0f5f74a7b05ddb 2024-08-20T22:08:09.9421420Z deleted: sha256:b1fc1302c8cd0f46187e82d20d88b8202fea499b545d365200cdf56252b58512 2024-08-20T22:08:09.9422382Z deleted: sha256:53b918ca72cbb58e7562e0c9dc2babc7518b268d0ec7536af7f2606e1dff95cd 2024-08-20T22:08:09.9423234Z deleted: sha256:4913bc3e68cdc276b1bdc93abfb9d4fe26c4edc0e02f58b976716f71c7cff383 2024-08-20T22:08:09.9424208Z deleted: sha256:12c37601a9c0fafe2eace184744245d4aacfb250075ccfea1318049fac59673c 2024-08-20T22:08:09.9425120Z deleted: sha256:76c1229c2e89d5a621f482e8ad46f1ab304af8101f6dc0c94e7a2d968d0e5dd1 2024-08-20T22:08:09.9425959Z deleted: sha256:8a0b862a9ff3652d53bc44d54db195bccd91824ae70c438bf13eeb29a262dae6 2024-08-20T22:08:09.9426800Z deleted: sha256:36bbacb75b1fc6aa5f9dd973767ed867786a74144cb10b11b3b5459b5a36453c 2024-08-20T22:08:09.9427637Z deleted: sha256:601fbf08be902110317ee896327aedc5f0d4f0bb2c68e3be8a9afab4d7e375ac 2024-08-20T22:08:09.9428503Z deleted: sha256:ebfd2b021b5e26b3ac1c81ab75ee84007f8ae82fc5329d1bb797589046ed7e3a 2024-08-20T22:08:09.9429346Z deleted: sha256:7f450dd7de6a0c0234061a4d6d27513dbd48d3af305d48ac844be6f213c9f484 2024-08-20T22:08:09.9430189Z deleted: sha256:0afb7f31e1ac43cfddfdc4034a31fa4ededc1d057ba90aa605105d5004d09ff6 2024-08-20T22:08:09.9431033Z deleted: sha256:83933310864f0db0b6271f6484fe2a2ec4c4aa107dc676707aab1a86e84e78b8 2024-08-20T22:08:09.9431846Z deleted: sha256:5a8dd28488303431ac01ac5ef13992e7bf21d61c6e94702046235c8e1892c01c 2024-08-20T22:08:09.9432669Z deleted: sha256:0d1932c19f3321c16b8b0febc6905cf104fe67e6cc7901d220de7f2899e7336c 2024-08-20T22:08:09.9433492Z deleted: sha256:701fa06264450ce0aa66e58d35aad330e7bd68b0fbac8949d8e2d7587340a139 2024-08-20T22:08:09.9434321Z deleted: sha256:f89a6b18745ab169f86e3cfa97cd85d114b480b991fc3d9c5be888a9c29bed91 2024-08-20T22:08:09.9435250Z deleted: sha256:174f5685490326fc0a1c0f5570b8663732189b327007e47ff13d2ca59673db02 2024-08-20T22:08:09.9435726Z 2024-08-20T22:08:09.9435874Z Total reclaimed space: 12.33GB 2024-08-20T22:08:09.9470875Z ##[group]Run set +e 2024-08-20T22:08:09.9471224Z set +e 2024-08-20T22:08:09.9471526Z set -x 2024-08-20T22:08:09.9471822Z  2024-08-20T22:08:09.9472139Z PT_DOMAIN=download.pytorch.org 2024-08-20T22:08:09.9472940Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T22:08:09.9474040Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T22:08:09.9474804Z # one is returned at random 2024-08-20T22:08:09.9475510Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T22:08:09.9476033Z  2024-08-20T22:08:09.9476363Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:08:09.9477001Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T22:08:09.9477786Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T22:08:09.9478348Z  2024-08-20T22:08:09.9478678Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:08:09.9479245Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T22:08:09.9479766Z  exit 1 2024-08-20T22:08:09.9480081Z  fi 2024-08-20T22:08:09.9480366Z fi 2024-08-20T22:08:09.9480641Z  2024-08-20T22:08:09.9481009Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T22:08:09.9481533Z  # Clean up any old records first 2024-08-20T22:08:09.9482064Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T22:08:09.9482528Z fi 2024-08-20T22:08:09.9482806Z  2024-08-20T22:08:09.9483253Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T22:08:09.9483818Z cat /etc/hosts 2024-08-20T22:08:09.9492831Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:09.9493332Z env: 2024-08-20T22:08:09.9493607Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:09.9493954Z ##[endgroup] 2024-08-20T22:08:09.9523337Z + PT_DOMAIN=download.pytorch.org 2024-08-20T22:08:09.9529015Z ++ dig -4 +short download.pytorch.org 2024-08-20T22:08:09.9529957Z ++ tail -n1 2024-08-20T22:08:09.9798391Z + RESOLVED_IP=18.160.10.22 2024-08-20T22:08:09.9798966Z + '[' -z 18.160.10.22 ']' 2024-08-20T22:08:09.9799440Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T22:08:09.9812094Z 108.138.64.61 download.pytorch.org 2024-08-20T22:08:09.9814565Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-20T22:08:10.1152304Z + echo '18.160.10.22 download.pytorch.org' 2024-08-20T22:08:10.1152858Z + sudo tee -a /etc/hosts 2024-08-20T22:08:10.1468886Z 18.160.10.22 download.pytorch.org 2024-08-20T22:08:10.1490537Z + cat /etc/hosts 2024-08-20T22:08:10.1501283Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T22:08:10.1513215Z ::1 localhost6 localhost6.localdomain6 2024-08-20T22:08:10.1513904Z 18.160.10.22 download.pytorch.org 2024-08-20T22:08:10.1669543Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T22:08:10.1670166Z with: 2024-08-20T22:08:10.1671302Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:10.1672572Z docker-build-dir: .ci/docker 2024-08-20T22:08:10.1672965Z working-directory: . 2024-08-20T22:08:10.1673446Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:10.1673983Z force-push: false 2024-08-20T22:08:10.1674296Z env: 2024-08-20T22:08:10.1674576Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:10.1674920Z ##[endgroup] 2024-08-20T22:08:10.1693838Z ##[group]Run set -ex 2024-08-20T22:08:10.1694207Z set -ex 2024-08-20T22:08:10.1694502Z  2024-08-20T22:08:10.1695069Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T22:08:10.1696070Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T22:08:10.1696898Z # job could then download the pre-built image as usual 2024-08-20T22:08:10.1697649Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T22:08:10.1698336Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:10.1698979Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:10.1699546Z  2024-08-20T22:08:10.1700062Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T22:08:10.1700702Z  exit 0 2024-08-20T22:08:10.1701017Z else 2024-08-20T22:08:10.1701383Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:10.1701930Z fi 2024-08-20T22:08:10.1702213Z  2024-08-20T22:08:10.1702688Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T22:08:10.1703559Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T22:08:10.1704357Z  # use it as it is, but first let's extract the tag 2024-08-20T22:08:10.1705068Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T22:08:10.1705802Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:10.1706517Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:10.1707080Z else 2024-08-20T22:08:10.1707522Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T22:08:10.1708196Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:10.1709123Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:10.1709888Z fi 2024-08-20T22:08:10.1719376Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:10.1720112Z env: 2024-08-20T22:08:10.1720508Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:10.1720963Z REPO_NAME: pytorch 2024-08-20T22:08:10.1722299Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:10.1723641Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:08:10.1724202Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:10.1724893Z ##[endgroup] 2024-08-20T22:08:10.1756462Z + [[ ! -d .ci/docker ]] 2024-08-20T22:08:10.1757190Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T22:08:10.1757701Z + echo skip=false 2024-08-20T22:08:10.1760046Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T22:08:10.1764518Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:10.1765818Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T22:08:10.1790055Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:10.1790958Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:10.1792582Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:10.1827692Z ##[group]Run set +e 2024-08-20T22:08:10.1828388Z set +e 2024-08-20T22:08:10.1828846Z set -x 2024-08-20T22:08:10.1829234Z  2024-08-20T22:08:10.1829600Z login() { 2024-08-20T22:08:10.1830499Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:08:10.1831329Z } 2024-08-20T22:08:10.1831694Z  2024-08-20T22:08:10.1832144Z retry () { 2024-08-20T22:08:10.1832640Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:08:10.1833277Z } 2024-08-20T22:08:10.1833613Z  2024-08-20T22:08:10.1834037Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:08:10.1834687Z  2024-08-20T22:08:10.1835291Z # Check if image already exists, if it does then skip building it 2024-08-20T22:08:10.1836213Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T22:08:10.1836901Z  exit 0 2024-08-20T22:08:10.1837270Z fi 2024-08-20T22:08:10.1837738Z  2024-08-20T22:08:10.1838409Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T22:08:10.1839386Z # be empty. The default action would be to continue rebuild the image 2024-08-20T22:08:10.1840291Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T22:08:10.1841130Z  # if we're on the base branch then use the parent commit 2024-08-20T22:08:10.1841863Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T22:08:10.1842594Z else 2024-08-20T22:08:10.1843141Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T22:08:10.1843975Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T22:08:10.1844643Z fi 2024-08-20T22:08:10.1845091Z  2024-08-20T22:08:10.1845503Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T22:08:10.1846175Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:10.1846760Z  2024-08-20T22:08:10.1847974Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T22:08:10.1848910Z  exit 0 2024-08-20T22:08:10.1849336Z fi 2024-08-20T22:08:10.1849711Z  2024-08-20T22:08:10.1850320Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T22:08:10.1851396Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T22:08:10.1852308Z  exit 1 2024-08-20T22:08:10.1852764Z fi 2024-08-20T22:08:10.1853222Z  2024-08-20T22:08:10.1853775Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T22:08:10.1854922Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T22:08:10.1855909Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T22:08:10.1857046Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T22:08:10.1858288Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T22:08:10.1859199Z fi 2024-08-20T22:08:10.1859662Z  2024-08-20T22:08:10.1860117Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:10.1869621Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:10.1870184Z env: 2024-08-20T22:08:10.1870550Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:10.1871093Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:08:10.1871634Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:08:10.1873060Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:10.1874499Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:10.1875342Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:10.1875986Z ##[endgroup] 2024-08-20T22:08:10.1909560Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:10.1910411Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:10.1912724Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:08:10.1913623Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:10.7316749Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:08:10.7317742Z Configure a credential helper to remove this warning. See 2024-08-20T22:08:10.7318897Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:08:10.7319579Z 2024-08-20T22:08:10.7319778Z Login Succeeded 2024-08-20T22:08:10.7343019Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:10.9589009Z { 2024-08-20T22:08:10.9589554Z "schemaVersion": 2, 2024-08-20T22:08:10.9590584Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T22:08:10.9591337Z "config": { 2024-08-20T22:08:10.9592050Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T22:08:10.9593032Z "size": 48711, 2024-08-20T22:08:10.9593864Z "digest": "sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb" 2024-08-20T22:08:10.9594855Z }, 2024-08-20T22:08:10.9595635Z "layers": [ 2024-08-20T22:08:10.9596136Z { 2024-08-20T22:08:10.9596845Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9597675Z "size": 28580681, 2024-08-20T22:08:10.9598909Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-20T22:08:10.9599801Z }, 2024-08-20T22:08:10.9600266Z { 2024-08-20T22:08:10.9600966Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9602044Z "size": 7943451, 2024-08-20T22:08:10.9602826Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-08-20T22:08:10.9603631Z }, 2024-08-20T22:08:10.9604062Z { 2024-08-20T22:08:10.9604585Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9605200Z "size": 55728572, 2024-08-20T22:08:10.9605917Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-08-20T22:08:10.9606622Z }, 2024-08-20T22:08:10.9606945Z { 2024-08-20T22:08:10.9607521Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9608172Z "size": 186, 2024-08-20T22:08:10.9608975Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-08-20T22:08:10.9609659Z }, 2024-08-20T22:08:10.9609994Z { 2024-08-20T22:08:10.9610853Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9611506Z "size": 6886, 2024-08-20T22:08:10.9612102Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-08-20T22:08:10.9612962Z }, 2024-08-20T22:08:10.9613574Z { 2024-08-20T22:08:10.9614098Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9614809Z "size": 1329001756, 2024-08-20T22:08:10.9615485Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-08-20T22:08:10.9616356Z }, 2024-08-20T22:08:10.9616735Z { 2024-08-20T22:08:10.9617327Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9618019Z "size": 62501, 2024-08-20T22:08:10.9618966Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-08-20T22:08:10.9619683Z }, 2024-08-20T22:08:10.9620100Z { 2024-08-20T22:08:10.9620664Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9621666Z "size": 1684, 2024-08-20T22:08:10.9622334Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-08-20T22:08:10.9623072Z }, 2024-08-20T22:08:10.9623449Z { 2024-08-20T22:08:10.9624063Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9624686Z "size": 1523, 2024-08-20T22:08:10.9625350Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-08-20T22:08:10.9626167Z }, 2024-08-20T22:08:10.9626475Z { 2024-08-20T22:08:10.9627022Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9627766Z "size": 2528295403, 2024-08-20T22:08:10.9628494Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-08-20T22:08:10.9629171Z }, 2024-08-20T22:08:10.9629601Z { 2024-08-20T22:08:10.9630142Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9630859Z "size": 86016, 2024-08-20T22:08:10.9631542Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-08-20T22:08:10.9632277Z }, 2024-08-20T22:08:10.9632649Z { 2024-08-20T22:08:10.9633273Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9633900Z "size": 1822, 2024-08-20T22:08:10.9634566Z "digest": "sha256:ac23c5fcc1ef310e9cd0d138b180281188e20a3e28c51cd14751622e3c7d92ea" 2024-08-20T22:08:10.9635367Z }, 2024-08-20T22:08:10.9635675Z { 2024-08-20T22:08:10.9636238Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9636956Z "size": 246761563, 2024-08-20T22:08:10.9637683Z "digest": "sha256:ca426ffc9d2bd051ad1ff5c4f02fd5d9be161e70953cc37882c815080c708f4d" 2024-08-20T22:08:10.9638376Z }, 2024-08-20T22:08:10.9638778Z { 2024-08-20T22:08:10.9639335Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9640051Z "size": 544, 2024-08-20T22:08:10.9640645Z "digest": "sha256:4373aa743caf6660c671295b837048c8350a2165f2336dc3b7b6c96462579e50" 2024-08-20T22:08:10.9641371Z }, 2024-08-20T22:08:10.9641768Z { 2024-08-20T22:08:10.9642268Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9642955Z "size": 1258, 2024-08-20T22:08:10.9643646Z "digest": "sha256:75b2abb16c685854c017d0cf75c626526c956734c656d9e78fdd4520317e9b3c" 2024-08-20T22:08:10.9644390Z }, 2024-08-20T22:08:10.9644724Z { 2024-08-20T22:08:10.9645307Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9645984Z "size": 484, 2024-08-20T22:08:10.9646614Z "digest": "sha256:ef1e3ef1e85f0124270b8506721219150e230184181f8207d64463bf5c67e656" 2024-08-20T22:08:10.9647604Z }, 2024-08-20T22:08:10.9647961Z { 2024-08-20T22:08:10.9648501Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9649207Z "size": 91711766, 2024-08-20T22:08:10.9649845Z "digest": "sha256:0bf61114d37b2d09065aa6852b1e78a32580a1ff011dd5d23f03b0f5c4e710af" 2024-08-20T22:08:10.9650568Z }, 2024-08-20T22:08:10.9650991Z { 2024-08-20T22:08:10.9651534Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9652223Z "size": 3390, 2024-08-20T22:08:10.9653115Z "digest": "sha256:2df849f72ba4258df10c9c45ace21a66d93b73196e78e96f12c13098822f60e2" 2024-08-20T22:08:10.9653839Z }, 2024-08-20T22:08:10.9654151Z { 2024-08-20T22:08:10.9654801Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9655461Z "size": 1909, 2024-08-20T22:08:10.9656123Z "digest": "sha256:34c47d63c5de27418739aa048b4dd781c3006d65691e9b903c95304d0509d89c" 2024-08-20T22:08:10.9656857Z }, 2024-08-20T22:08:10.9657207Z { 2024-08-20T22:08:10.9657770Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9658524Z "size": 701, 2024-08-20T22:08:10.9659172Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:10.9660117Z }, 2024-08-20T22:08:10.9660526Z { 2024-08-20T22:08:10.9661073Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9661881Z "size": 2864254879, 2024-08-20T22:08:10.9662623Z "digest": "sha256:6efb96fc87ea1bca54876d85331e5585e28f325a1d60ee1aef7811eabb74f933" 2024-08-20T22:08:10.9663383Z }, 2024-08-20T22:08:10.9663695Z { 2024-08-20T22:08:10.9664284Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9664961Z "size": 380, 2024-08-20T22:08:10.9665674Z "digest": "sha256:b119c4fc6c58d9e72744201ff4850b34184a94eb3566031b7d36e87ecce950a6" 2024-08-20T22:08:10.9666367Z }, 2024-08-20T22:08:10.9666748Z { 2024-08-20T22:08:10.9667338Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9667989Z "size": 12900, 2024-08-20T22:08:10.9668664Z "digest": "sha256:33b639226eeefa41d7bb26a8ecc20d271ceef45571dabfd80aaee3634926381b" 2024-08-20T22:08:10.9669458Z }, 2024-08-20T22:08:10.9669838Z { 2024-08-20T22:08:10.9670339Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9671061Z "size": 803, 2024-08-20T22:08:10.9671740Z "digest": "sha256:d3547bd43146930aa830d54562c98aa678e355cf5f84527e9717eb72fffe8ca3" 2024-08-20T22:08:10.9672467Z }, 2024-08-20T22:08:10.9672866Z { 2024-08-20T22:08:10.9673446Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9674113Z "size": 106, 2024-08-20T22:08:10.9674848Z "digest": "sha256:5645520256fa6d7f2270608eaed53849a131d03fbf4c5d4472c9cfde1a439e47" 2024-08-20T22:08:10.9675538Z }, 2024-08-20T22:08:10.9675892Z { 2024-08-20T22:08:10.9676522Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9677148Z "size": 503, 2024-08-20T22:08:10.9677799Z "digest": "sha256:829ed7d91e3818d0e11834c71551117c7aeec0a51d798e93d641faa0c02582cb" 2024-08-20T22:08:10.9678601Z }, 2024-08-20T22:08:10.9678965Z { 2024-08-20T22:08:10.9679475Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9680246Z "size": 121477267, 2024-08-20T22:08:10.9680907Z "digest": "sha256:385d0421d67c533d628985c453a186028a61f6ad4209ae519ec3d4ecea661d3e" 2024-08-20T22:08:10.9681646Z }, 2024-08-20T22:08:10.9682000Z { 2024-08-20T22:08:10.9682536Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9683223Z "size": 109, 2024-08-20T22:08:10.9683887Z "digest": "sha256:7cbfce999d76851e07dd71056a6d852629d789ccfd46be44cc405b451fb8bb58" 2024-08-20T22:08:10.9684615Z }, 2024-08-20T22:08:10.9684985Z { 2024-08-20T22:08:10.9685568Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9686185Z "size": 490, 2024-08-20T22:08:10.9686944Z "digest": "sha256:6658e558ebe496f0d628947534b0ea1e44808f84f2dfd8a9a704e8aa5329d791" 2024-08-20T22:08:10.9687708Z }, 2024-08-20T22:08:10.9688074Z { 2024-08-20T22:08:10.9688584Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9689346Z "size": 297, 2024-08-20T22:08:10.9690006Z "digest": "sha256:33444a5a7e01a3010fd1cc16b32a66892a08600d7f700747d23faf3c36624f3a" 2024-08-20T22:08:10.9690780Z }, 2024-08-20T22:08:10.9691214Z { 2024-08-20T22:08:10.9691735Z + exit 0 2024-08-20T22:08:10.9692308Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9692986Z "size": 103, 2024-08-20T22:08:10.9693678Z "digest": "sha256:7c47122f4e09b47e891b4ae9e499a35a4eb46faacae87d0e946615adda6e0706" 2024-08-20T22:08:10.9694463Z }, 2024-08-20T22:08:10.9694806Z { 2024-08-20T22:08:10.9695339Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9696058Z "size": 1473, 2024-08-20T22:08:10.9696670Z "digest": "sha256:1811ff4d3556ac255604f346d879528aff2aadd7652c2d39baf4fe6eaa60aec3" 2024-08-20T22:08:10.9697396Z }, 2024-08-20T22:08:10.9697810Z { 2024-08-20T22:08:10.9698436Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9699077Z "size": 594729516, 2024-08-20T22:08:10.9699789Z "digest": "sha256:7754ee068b33f760d9c320c26a1a3935247ace0c8c463a976105c1586a43fb1c" 2024-08-20T22:08:10.9700531Z }, 2024-08-20T22:08:10.9700866Z { 2024-08-20T22:08:10.9701552Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9702209Z "size": 164, 2024-08-20T22:08:10.9702872Z "digest": "sha256:0479fbac3361713c075bf788e778117917dcacdd1ff71ed79a789dd2016a1714" 2024-08-20T22:08:10.9703606Z }, 2024-08-20T22:08:10.9703979Z { 2024-08-20T22:08:10.9704511Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9705212Z "size": 564, 2024-08-20T22:08:10.9705830Z "digest": "sha256:f1d6874dee9c34d154278d2b6935d04447f07fa89d25d4a09398234040d8a7e8" 2024-08-20T22:08:10.9706545Z }, 2024-08-20T22:08:10.9706966Z { 2024-08-20T22:08:10.9707490Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9708159Z "size": 35874203, 2024-08-20T22:08:10.9708885Z "digest": "sha256:afc429e87b593a64f3e0a1be51e09e966ea1aa71e6e8a0bec4edc9f2f7bc1c05" 2024-08-20T22:08:10.9709642Z }, 2024-08-20T22:08:10.9709950Z { 2024-08-20T22:08:10.9710554Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9711210Z "size": 104, 2024-08-20T22:08:10.9711889Z "digest": "sha256:019620471c2592cd22437fe8ed1a94c784114cf7445c5c2ab36dd0bae727fc66" 2024-08-20T22:08:10.9712591Z }, 2024-08-20T22:08:10.9712940Z { 2024-08-20T22:08:10.9713520Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9714228Z "size": 425, 2024-08-20T22:08:10.9714856Z "digest": "sha256:27aeb1b2cd5e28d8f0e54fc1090baf88abc29ebf5fce088b493fcae99c9550d1" 2024-08-20T22:08:10.9715647Z }, 2024-08-20T22:08:10.9716020Z { 2024-08-20T22:08:10.9716522Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9717264Z "size": 20262183, 2024-08-20T22:08:10.9717931Z "digest": "sha256:829c0e19f4b7797e9120a86172fabfd710f9b3971ffd714a616931a3fadc7192" 2024-08-20T22:08:10.9718701Z }, 2024-08-20T22:08:10.9719088Z { 2024-08-20T22:08:10.9719632Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9720295Z "size": 438, 2024-08-20T22:08:10.9721044Z "digest": "sha256:560cd6122a3868e7e82e75eb103d182edf6512114d61ccdfae099af00cb16b4c" 2024-08-20T22:08:10.9721734Z }, 2024-08-20T22:08:10.9722081Z { 2024-08-20T22:08:10.9722701Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9723356Z "size": 701, 2024-08-20T22:08:10.9723988Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:10.9724762Z }, 2024-08-20T22:08:10.9725112Z { 2024-08-20T22:08:10.9725636Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9726342Z "size": 141, 2024-08-20T22:08:10.9727006Z "digest": "sha256:5a4eb19d72198a01f3babd4711c40fcf7c6e5cb9d5cfedc507b0061408d47097" 2024-08-20T22:08:10.9727757Z }, 2024-08-20T22:08:10.9728144Z { 2024-08-20T22:08:10.9728688Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9729478Z "size": 135, 2024-08-20T22:08:10.9730173Z "digest": "sha256:da6a838a66c72b5de9a5590c233779880ea901c1264940a1c618825d2501ffd1" 2024-08-20T22:08:10.9730867Z }, 2024-08-20T22:08:10.9731211Z { 2024-08-20T22:08:10.9731793Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9732440Z "size": 4838068375, 2024-08-20T22:08:10.9733115Z "digest": "sha256:cbb79a7053537bed2d80b77198ea552b985563b11d3f8616593c6ab575a3a090" 2024-08-20T22:08:10.9733918Z }, 2024-08-20T22:08:10.9734272Z { 2024-08-20T22:08:10.9734802Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9735537Z "size": 194, 2024-08-20T22:08:10.9736187Z "digest": "sha256:4d4fa6802ab3e6b36d8d4adb650046e9370c4a16a19876970559ef8a312ca855" 2024-08-20T22:08:10.9737075Z }, 2024-08-20T22:08:10.9737384Z { 2024-08-20T22:08:10.9737922Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9738657Z "size": 564, 2024-08-20T22:08:10.9739317Z "digest": "sha256:0f085d4fa279366fcd52d303b7ad8abddc5962a8872174b8109f3f5d5ca5f2cf" 2024-08-20T22:08:10.9740005Z }, 2024-08-20T22:08:10.9740425Z { 2024-08-20T22:08:10.9740960Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9741698Z "size": 43162921, 2024-08-20T22:08:10.9742460Z "digest": "sha256:4a0412373305517489d891a4749e6d2b80b4aeb7df3b8adca0ed0abb2f3e5300" 2024-08-20T22:08:10.9743180Z }, 2024-08-20T22:08:10.9743548Z { 2024-08-20T22:08:10.9744099Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9744757Z "size": 106, 2024-08-20T22:08:10.9745419Z "digest": "sha256:302c8c1779444e54919f83e91c6d5e1fdea653fc899be802b3a2a34d34d99d5d" 2024-08-20T22:08:10.9746207Z }, 2024-08-20T22:08:10.9746515Z { 2024-08-20T22:08:10.9747229Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9748023Z "size": 1380, 2024-08-20T22:08:10.9748759Z "digest": "sha256:02970347dff622ee45629bc28788716d00528464f71f88ec7b74ae5f44a486e2" 2024-08-20T22:08:10.9749412Z }, 2024-08-20T22:08:10.9749809Z { 2024-08-20T22:08:10.9750368Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9750999Z "size": 701, 2024-08-20T22:08:10.9751696Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:10.9752437Z }, 2024-08-20T22:08:10.9752832Z { 2024-08-20T22:08:10.9753335Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9754013Z "size": 139, 2024-08-20T22:08:10.9754697Z "digest": "sha256:916c5e8580164926a3aec87e88a5d1c2d02756e49fdeed4198e1e6304efa2fd3" 2024-08-20T22:08:10.9755443Z }, 2024-08-20T22:08:10.9755784Z { 2024-08-20T22:08:10.9756373Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9757059Z "size": 118, 2024-08-20T22:08:10.9757668Z "digest": "sha256:c42b3602a1fb4df5111224650f15adffb025979ae9c614807379ea84e189533b" 2024-08-20T22:08:10.9758437Z }, 2024-08-20T22:08:10.9758802Z { 2024-08-20T22:08:10.9759336Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9760001Z "size": 4324672728, 2024-08-20T22:08:10.9760699Z "digest": "sha256:11fad972ef6cb9d2dc236504b0ee5b6e5ea996e4576ece0ed3262fbb9ee2de60" 2024-08-20T22:08:10.9761440Z }, 2024-08-20T22:08:10.9761860Z { 2024-08-20T22:08:10.9762364Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9763052Z "size": 174, 2024-08-20T22:08:10.9763764Z "digest": "sha256:f957c16244819132b478272aaf24231699c4fcbd495c229a574f94daab8d6b86" 2024-08-20T22:08:10.9764481Z }, 2024-08-20T22:08:10.9764798Z { 2024-08-20T22:08:10.9765402Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9766071Z "size": 907, 2024-08-20T22:08:10.9766695Z "digest": "sha256:768c31dca016b80bbdc973c8ee818034bdbd17b7ea9bdadbf0ec45ea51005720" 2024-08-20T22:08:10.9767653Z }, 2024-08-20T22:08:10.9768004Z { 2024-08-20T22:08:10.9768560Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9769242Z "size": 701, 2024-08-20T22:08:10.9769927Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:10.9770682Z }, 2024-08-20T22:08:10.9771101Z { 2024-08-20T22:08:10.9771606Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9772284Z "size": 134, 2024-08-20T22:08:10.9772998Z "digest": "sha256:8c32d0ecebbe2054c96a3a5c1f3d6c8b87c2e826686b369e5242cd55d0f01c2d" 2024-08-20T22:08:10.9773720Z }, 2024-08-20T22:08:10.9774077Z { 2024-08-20T22:08:10.9774670Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9775475Z "size": 32, 2024-08-20T22:08:10.9776092Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:10.9776924Z }, 2024-08-20T22:08:10.9777280Z { 2024-08-20T22:08:10.9777871Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9778511Z "size": 158, 2024-08-20T22:08:10.9779169Z "digest": "sha256:78ee77eddef23f20afc78e2924f4467041cadb754634ee3df431c68211cf676d" 2024-08-20T22:08:10.9779952Z }, 2024-08-20T22:08:10.9780322Z { 2024-08-20T22:08:10.9780817Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9781615Z "size": 1841, 2024-08-20T22:08:10.9782282Z "digest": "sha256:37513e9b43d7123e7ddee09100f62a72b57af7b3833eaecd515e247e2f63e13f" 2024-08-20T22:08:10.9782971Z }, 2024-08-20T22:08:10.9783385Z { 2024-08-20T22:08:10.9783959Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9784632Z "size": 7529776, 2024-08-20T22:08:10.9785335Z "digest": "sha256:a7a5392e2bf6b73d9be292cc22a99ad4637636c15733beb7afbe6403b5ba21a8" 2024-08-20T22:08:10.9786070Z }, 2024-08-20T22:08:10.9786423Z { 2024-08-20T22:08:10.9787045Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9787673Z "size": 163, 2024-08-20T22:08:10.9788310Z "digest": "sha256:b10de02255c021b2366b85bf12ddf593aac89f88964201c455b71a145bc5ca2e" 2024-08-20T22:08:10.9789144Z }, 2024-08-20T22:08:10.9789448Z { 2024-08-20T22:08:10.9790002Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9790747Z "size": 7943, 2024-08-20T22:08:10.9791406Z "digest": "sha256:e56eeff2db93ffe5002a2bb764795b7333321c6802cc28bef0801a619fe6a07e" 2024-08-20T22:08:10.9792113Z }, 2024-08-20T22:08:10.9792519Z { 2024-08-20T22:08:10.9793060Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9793752Z "size": 8068, 2024-08-20T22:08:10.9794430Z "digest": "sha256:70ac3e38d0fd1f0f46ad710c3fc485fd01406606054e8cf84a63af1570bc9f3a" 2024-08-20T22:08:10.9795154Z }, 2024-08-20T22:08:10.9795520Z { 2024-08-20T22:08:10.9796117Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9796764Z "size": 302, 2024-08-20T22:08:10.9797407Z "digest": "sha256:eb09cbf3e4168749cc8605e3188ed35070f6037d13930e443c38d8941734488f" 2024-08-20T22:08:10.9798216Z }, 2024-08-20T22:08:10.9798546Z { 2024-08-20T22:08:10.9799094Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9799807Z "size": 7629638, 2024-08-20T22:08:10.9800479Z "digest": "sha256:ee1273153675e6543b58f5be50321abee5e91f7351701d5d501a6837eaf676ef" 2024-08-20T22:08:10.9801148Z }, 2024-08-20T22:08:10.9801542Z { 2024-08-20T22:08:10.9802101Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9802810Z "size": 108, 2024-08-20T22:08:10.9803458Z "digest": "sha256:e22979f9e0eedfa16f98eb4fa26f30a50c237f5ffd2f6e5b898f2b908058496d" 2024-08-20T22:08:10.9804200Z }, 2024-08-20T22:08:10.9804640Z { 2024-08-20T22:08:10.9805161Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9805926Z "size": 54145661, 2024-08-20T22:08:10.9806685Z "digest": "sha256:6c1f0aabce7c84f9b0b9ab3b96325778dba1c1465b7463c5b4c6be1cf52b74db" 2024-08-20T22:08:10.9807413Z }, 2024-08-20T22:08:10.9807719Z { 2024-08-20T22:08:10.9808319Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9808975Z "size": 473, 2024-08-20T22:08:10.9809612Z "digest": "sha256:7ebe49c3a23e71b289ce012b48dad35791a2f2aa7fcee2326a64d763b0d14df8" 2024-08-20T22:08:10.9810366Z }, 2024-08-20T22:08:10.9810706Z { 2024-08-20T22:08:10.9811273Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9812001Z "size": 1374858766, 2024-08-20T22:08:10.9812634Z "digest": "sha256:5d115b29b5d53e5418762bc41ae975dd245bcffc3d058b8902c58146d505cb37" 2024-08-20T22:08:10.9813513Z }, 2024-08-20T22:08:10.9813919Z { 2024-08-20T22:08:10.9814434Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9815141Z "size": 106, 2024-08-20T22:08:10.9815891Z "digest": "sha256:7f2d7a04fbe65bbe1e895138e0cfeeef53ad926c890bab8e7e132ad1e655e48f" 2024-08-20T22:08:10.9816697Z }, 2024-08-20T22:08:10.9817008Z { 2024-08-20T22:08:10.9817634Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:10.9818386Z "size": 559, 2024-08-20T22:08:10.9819066Z "digest": "sha256:d79fd7d526989e554455eda7d459f2c88444030f61710101e52ec8c7a49b7b79" 2024-08-20T22:08:10.9819863Z }, 2024-08-20T22:08:10.9820224Z { 2024-08-20T22:08:11.0008062Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:11.0008784Z "size": 46248488, 2024-08-20T22:08:11.0009360Z "digest": "sha256:9752d6fbbcab4e7403f26777c5b65911320de6389c81b833a47ba45c4f3354b4" 2024-08-20T22:08:11.0010022Z }, 2024-08-20T22:08:11.0010277Z { 2024-08-20T22:08:11.0010733Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:11.0011307Z "size": 111, 2024-08-20T22:08:11.0011857Z "digest": "sha256:83808b2eff4d71ed46db7c29f66fe918f328405c25b314225dfad01b4868d1f4" 2024-08-20T22:08:11.0012499Z }, 2024-08-20T22:08:11.0012755Z { 2024-08-20T22:08:11.0013202Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:11.0013768Z "size": 32, 2024-08-20T22:08:11.0014326Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:11.0014954Z }, 2024-08-20T22:08:11.0015200Z { 2024-08-20T22:08:11.0015636Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:11.0016203Z "size": 32, 2024-08-20T22:08:11.0016748Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:11.0017375Z }, 2024-08-20T22:08:11.0017621Z { 2024-08-20T22:08:11.0018055Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:11.0018624Z "size": 32, 2024-08-20T22:08:11.0019161Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:11.0019793Z }, 2024-08-20T22:08:11.0020040Z { 2024-08-20T22:08:11.0020471Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:11.0021032Z "size": 32, 2024-08-20T22:08:11.0021700Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:11.0022334Z } 2024-08-20T22:08:11.0022574Z ] 2024-08-20T22:08:11.0022822Z } 2024-08-20T22:08:11.0130766Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:08:11.0131267Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:08:11.0131849Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T22:08:11.0141035Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:11.0141650Z env: 2024-08-20T22:08:11.0141932Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:11.0143145Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:11.0144369Z ##[endgroup] 2024-08-20T22:08:11.0176650Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:11.0228443Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T22:08:11.0229035Z with: 2024-08-20T22:08:11.0230134Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:11.0231489Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:11.0232016Z env: 2024-08-20T22:08:11.0232297Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:11.0232833Z ##[endgroup] 2024-08-20T22:08:11.0250496Z ##[group]Run set -x 2024-08-20T22:08:11.0250839Z set -x 2024-08-20T22:08:11.0251143Z set +e 2024-08-20T22:08:11.0251437Z  2024-08-20T22:08:11.0251711Z login() { 2024-08-20T22:08:11.0252388Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:08:11.0253119Z } 2024-08-20T22:08:11.0253400Z  2024-08-20T22:08:11.0253707Z retry () { 2024-08-20T22:08:11.0254111Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:08:11.0254567Z } 2024-08-20T22:08:11.0254850Z  2024-08-20T22:08:11.0255179Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:08:11.0255600Z  2024-08-20T22:08:11.0255879Z set -e 2024-08-20T22:08:11.0256362Z # ignore output since only exit code is used for conditional 2024-08-20T22:08:11.0257098Z # only pull docker image if it's not available locally 2024-08-20T22:08:11.0257881Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T22:08:11.0258645Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T22:08:11.0259097Z fi 2024-08-20T22:08:11.0267205Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:11.0267713Z env: 2024-08-20T22:08:11.0268005Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:11.0269180Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:11.0270517Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:11.0271046Z ##[endgroup] 2024-08-20T22:08:11.0299727Z + set +e 2024-08-20T22:08:11.0300404Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:11.0301216Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:11.0304191Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:08:11.0306333Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:11.5692891Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:08:11.5693806Z Configure a credential helper to remove this warning. See 2024-08-20T22:08:11.5694866Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:08:11.5695425Z 2024-08-20T22:08:11.5695563Z Login Succeeded 2024-08-20T22:08:11.5716352Z + set -e 2024-08-20T22:08:11.5717691Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:11.5881528Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:11.5883791Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:11.8190106Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-08-20T22:08:11.8197166Z 7a2c55901189: Pulling fs layer 2024-08-20T22:08:11.8197728Z 224fe954d725: Pulling fs layer 2024-08-20T22:08:11.8198587Z 75722010b82e: Pulling fs layer 2024-08-20T22:08:11.8199143Z d527cbbb87e3: Pulling fs layer 2024-08-20T22:08:11.8199659Z b57676e46aee: Pulling fs layer 2024-08-20T22:08:11.8200189Z a8c1e85b5e14: Pulling fs layer 2024-08-20T22:08:11.8200709Z a41a8d1c11c8: Pulling fs layer 2024-08-20T22:08:11.8201247Z 0c1227890755: Pulling fs layer 2024-08-20T22:08:11.8201756Z d8d1234baab3: Pulling fs layer 2024-08-20T22:08:11.8202271Z 7ed32bc8e469: Pulling fs layer 2024-08-20T22:08:11.8202801Z ec1e7978c1fe: Pulling fs layer 2024-08-20T22:08:11.8203544Z ac23c5fcc1ef: Pulling fs layer 2024-08-20T22:08:11.8203969Z ca426ffc9d2b: Pulling fs layer 2024-08-20T22:08:11.8204439Z 4373aa743caf: Pulling fs layer 2024-08-20T22:08:11.8204963Z 75b2abb16c68: Pulling fs layer 2024-08-20T22:08:11.8205475Z ef1e3ef1e85f: Pulling fs layer 2024-08-20T22:08:11.8205998Z 0bf61114d37b: Pulling fs layer 2024-08-20T22:08:11.8206521Z 2df849f72ba4: Pulling fs layer 2024-08-20T22:08:11.8207057Z 34c47d63c5de: Pulling fs layer 2024-08-20T22:08:11.8207556Z d527cbbb87e3: Waiting 2024-08-20T22:08:11.8208032Z 1fef4bd19792: Pulling fs layer 2024-08-20T22:08:11.8208603Z d8d1234baab3: Waiting 2024-08-20T22:08:11.8208995Z 6efb96fc87ea: Pulling fs layer 2024-08-20T22:08:11.8209507Z b57676e46aee: Waiting 2024-08-20T22:08:11.8209975Z b119c4fc6c58: Pulling fs layer 2024-08-20T22:08:11.8210505Z 33b639226eee: Pulling fs layer 2024-08-20T22:08:11.8210984Z 7ed32bc8e469: Waiting 2024-08-20T22:08:11.8211332Z d3547bd43146: Pulling fs layer 2024-08-20T22:08:11.8211712Z a8c1e85b5e14: Waiting 2024-08-20T22:08:11.8212049Z 5645520256fa: Pulling fs layer 2024-08-20T22:08:11.8212436Z 829ed7d91e38: Pulling fs layer 2024-08-20T22:08:11.8212824Z 385d0421d67c: Pulling fs layer 2024-08-20T22:08:11.8213227Z ec1e7978c1fe: Waiting 2024-08-20T22:08:11.8213553Z a41a8d1c11c8: Waiting 2024-08-20T22:08:11.8213864Z 0c1227890755: Waiting 2024-08-20T22:08:11.8214186Z ac23c5fcc1ef: Waiting 2024-08-20T22:08:11.8214508Z ca426ffc9d2b: Waiting 2024-08-20T22:08:11.8214833Z 4373aa743caf: Waiting 2024-08-20T22:08:11.8215153Z 34c47d63c5de: Waiting 2024-08-20T22:08:11.8215490Z 7cbfce999d76: Pulling fs layer 2024-08-20T22:08:11.8215878Z 6658e558ebe4: Pulling fs layer 2024-08-20T22:08:11.8216253Z 33444a5a7e01: Pulling fs layer 2024-08-20T22:08:11.8216632Z 7c47122f4e09: Pulling fs layer 2024-08-20T22:08:11.8217012Z 1811ff4d3556: Pulling fs layer 2024-08-20T22:08:11.8217377Z 0bf61114d37b: Waiting 2024-08-20T22:08:11.8217695Z ef1e3ef1e85f: Waiting 2024-08-20T22:08:11.8218024Z 2df849f72ba4: Waiting 2024-08-20T22:08:11.8218354Z 6efb96fc87ea: Waiting 2024-08-20T22:08:11.8218692Z 7754ee068b33: Pulling fs layer 2024-08-20T22:08:11.8219048Z 33b639226eee: Waiting 2024-08-20T22:08:11.8219383Z 0479fbac3361: Pulling fs layer 2024-08-20T22:08:11.8219768Z f1d6874dee9c: Pulling fs layer 2024-08-20T22:08:11.8220147Z afc429e87b59: Pulling fs layer 2024-08-20T22:08:11.8220513Z b119c4fc6c58: Waiting 2024-08-20T22:08:11.8220846Z 019620471c25: Pulling fs layer 2024-08-20T22:08:11.8221226Z 27aeb1b2cd5e: Pulling fs layer 2024-08-20T22:08:11.8221723Z 6658e558ebe4: Waiting 2024-08-20T22:08:11.8222061Z 829c0e19f4b7: Pulling fs layer 2024-08-20T22:08:11.8222425Z 385d0421d67c: Waiting 2024-08-20T22:08:11.8222758Z 560cd6122a38: Pulling fs layer 2024-08-20T22:08:11.8223118Z 1811ff4d3556: Waiting 2024-08-20T22:08:11.8223456Z 5a4eb19d7219: Pulling fs layer 2024-08-20T22:08:11.8223833Z 7754ee068b33: Waiting 2024-08-20T22:08:11.8224163Z da6a838a66c7: Pulling fs layer 2024-08-20T22:08:11.8224548Z cbb79a705353: Pulling fs layer 2024-08-20T22:08:11.8224923Z 0479fbac3361: Waiting 2024-08-20T22:08:11.8225260Z 4d4fa6802ab3: Pulling fs layer 2024-08-20T22:08:11.8225620Z 5645520256fa: Waiting 2024-08-20T22:08:11.8225952Z 0f085d4fa279: Pulling fs layer 2024-08-20T22:08:11.8226318Z 829ed7d91e38: Waiting 2024-08-20T22:08:11.8226646Z 4a0412373305: Pulling fs layer 2024-08-20T22:08:11.8227021Z 302c8c177944: Pulling fs layer 2024-08-20T22:08:11.8227396Z 02970347dff6: Pulling fs layer 2024-08-20T22:08:11.8227758Z afc429e87b59: Waiting 2024-08-20T22:08:11.8228251Z 916c5e858016: Pulling fs layer 2024-08-20T22:08:11.8228642Z c42b3602a1fb: Pulling fs layer 2024-08-20T22:08:11.8229009Z 829c0e19f4b7: Waiting 2024-08-20T22:08:11.8229328Z 019620471c25: Waiting 2024-08-20T22:08:11.8229656Z 11fad972ef6c: Pulling fs layer 2024-08-20T22:08:11.8230025Z f1d6874dee9c: Waiting 2024-08-20T22:08:11.8230343Z 7c47122f4e09: Waiting 2024-08-20T22:08:11.8230670Z f957c1624481: Pulling fs layer 2024-08-20T22:08:11.8231034Z 560cd6122a38: Waiting 2024-08-20T22:08:11.8231350Z 75b2abb16c68: Waiting 2024-08-20T22:08:11.8231769Z 768c31dca016: Pulling fs layer 2024-08-20T22:08:11.8232136Z 302c8c177944: Waiting 2024-08-20T22:08:11.8232453Z da6a838a66c7: Waiting 2024-08-20T22:08:11.8232790Z 8c32d0ecebbe: Pulling fs layer 2024-08-20T22:08:11.8233172Z 4f4fb700ef54: Pulling fs layer 2024-08-20T22:08:11.8233550Z 78ee77eddef2: Pulling fs layer 2024-08-20T22:08:11.8233924Z 5a4eb19d7219: Waiting 2024-08-20T22:08:11.8234242Z f957c1624481: Waiting 2024-08-20T22:08:11.8234571Z 37513e9b43d7: Pulling fs layer 2024-08-20T22:08:11.8234941Z 0f085d4fa279: Waiting 2024-08-20T22:08:11.8235274Z a7a5392e2bf6: Pulling fs layer 2024-08-20T22:08:11.8235647Z 768c31dca016: Waiting 2024-08-20T22:08:11.8235975Z b10de02255c0: Pulling fs layer 2024-08-20T22:08:11.8236342Z 11fad972ef6c: Waiting 2024-08-20T22:08:11.8236682Z e56eeff2db93: Pulling fs layer 2024-08-20T22:08:11.8237065Z 70ac3e38d0fd: Pulling fs layer 2024-08-20T22:08:11.8237433Z 916c5e858016: Waiting 2024-08-20T22:08:11.8237769Z eb09cbf3e416: Pulling fs layer 2024-08-20T22:08:11.8238149Z c42b3602a1fb: Waiting 2024-08-20T22:08:11.8238528Z ee1273153675: Pulling fs layer 2024-08-20T22:08:11.8238905Z e22979f9e0ee: Pulling fs layer 2024-08-20T22:08:11.8239273Z b10de02255c0: Waiting 2024-08-20T22:08:11.8239609Z 6c1f0aabce7c: Pulling fs layer 2024-08-20T22:08:11.8239968Z 4a0412373305: Waiting 2024-08-20T22:08:11.8240287Z 70ac3e38d0fd: Waiting 2024-08-20T22:08:11.8240625Z 7ebe49c3a23e: Pulling fs layer 2024-08-20T22:08:11.8241001Z 5d115b29b5d5: Pulling fs layer 2024-08-20T22:08:11.8241377Z eb09cbf3e416: Waiting 2024-08-20T22:08:11.8241719Z 7f2d7a04fbe6: Pulling fs layer 2024-08-20T22:08:11.8242092Z e56eeff2db93: Waiting 2024-08-20T22:08:11.8242412Z 78ee77eddef2: Waiting 2024-08-20T22:08:11.8242748Z d79fd7d52698: Pulling fs layer 2024-08-20T22:08:11.8243134Z 9752d6fbbcab: Pulling fs layer 2024-08-20T22:08:11.8243515Z 83808b2eff4d: Pulling fs layer 2024-08-20T22:08:11.8243876Z 7ebe49c3a23e: Waiting 2024-08-20T22:08:11.8244199Z 7f2d7a04fbe6: Waiting 2024-08-20T22:08:11.8244525Z 4f4fb700ef54: Waiting 2024-08-20T22:08:11.8244841Z 8c32d0ecebbe: Waiting 2024-08-20T22:08:11.8245161Z ee1273153675: Waiting 2024-08-20T22:08:11.8245479Z 6c1f0aabce7c: Waiting 2024-08-20T22:08:11.8245813Z 83808b2eff4d: Waiting 2024-08-20T22:08:11.8246127Z 9752d6fbbcab: Waiting 2024-08-20T22:08:11.8246448Z a7a5392e2bf6: Waiting 2024-08-20T22:08:11.8246766Z 1fef4bd19792: Waiting 2024-08-20T22:08:11.8247469Z 37513e9b43d7: Waiting 2024-08-20T22:08:11.8247822Z cbb79a705353: Waiting 2024-08-20T22:08:11.9550354Z 224fe954d725: Verifying Checksum 2024-08-20T22:08:11.9550779Z 224fe954d725: Download complete 2024-08-20T22:08:12.0299909Z d527cbbb87e3: Download complete 2024-08-20T22:08:12.1076583Z b57676e46aee: Verifying Checksum 2024-08-20T22:08:12.1077122Z b57676e46aee: Download complete 2024-08-20T22:08:12.1616874Z 7a2c55901189: Verifying Checksum 2024-08-20T22:08:12.1617379Z 7a2c55901189: Download complete 2024-08-20T22:08:12.2479335Z a41a8d1c11c8: Verifying Checksum 2024-08-20T22:08:12.2479842Z a41a8d1c11c8: Download complete 2024-08-20T22:08:12.3230693Z 0c1227890755: Download complete 2024-08-20T22:08:12.3960727Z d8d1234baab3: Verifying Checksum 2024-08-20T22:08:12.3961434Z d8d1234baab3: Download complete 2024-08-20T22:08:12.4203740Z 75722010b82e: Verifying Checksum 2024-08-20T22:08:12.4204316Z 75722010b82e: Download complete 2024-08-20T22:08:12.4887984Z ec1e7978c1fe: Verifying Checksum 2024-08-20T22:08:12.4888793Z ec1e7978c1fe: Download complete 2024-08-20T22:08:12.5615502Z ac23c5fcc1ef: Download complete 2024-08-20T22:08:13.3097965Z 7a2c55901189: Pull complete 2024-08-20T22:08:13.6157929Z 224fe954d725: Pull complete 2024-08-20T22:08:14.5516032Z 75722010b82e: Pull complete 2024-08-20T22:08:14.5662037Z d527cbbb87e3: Pull complete 2024-08-20T22:08:14.5838562Z b57676e46aee: Pull complete 2024-08-20T22:08:15.0785516Z ca426ffc9d2b: Verifying Checksum 2024-08-20T22:08:15.0786150Z ca426ffc9d2b: Download complete 2024-08-20T22:08:15.1657936Z 4373aa743caf: Download complete 2024-08-20T22:08:15.2322398Z 75b2abb16c68: Verifying Checksum 2024-08-20T22:08:15.2323232Z 75b2abb16c68: Download complete 2024-08-20T22:08:15.3199079Z ef1e3ef1e85f: Download complete 2024-08-20T22:08:16.2860985Z 0bf61114d37b: Verifying Checksum 2024-08-20T22:08:16.2861775Z 0bf61114d37b: Download complete 2024-08-20T22:08:16.3623570Z 2df849f72ba4: Verifying Checksum 2024-08-20T22:08:16.3624161Z 2df849f72ba4: Download complete 2024-08-20T22:08:16.4575775Z 34c47d63c5de: Verifying Checksum 2024-08-20T22:08:16.4576301Z 34c47d63c5de: Download complete 2024-08-20T22:08:16.5353142Z 1fef4bd19792: Verifying Checksum 2024-08-20T22:08:16.5354398Z 1fef4bd19792: Download complete 2024-08-20T22:08:25.4991134Z a8c1e85b5e14: Download complete 2024-08-20T22:08:25.5591364Z b119c4fc6c58: Verifying Checksum 2024-08-20T22:08:25.5592053Z b119c4fc6c58: Download complete 2024-08-20T22:08:25.6190092Z 33b639226eee: Download complete 2024-08-20T22:08:25.6783655Z d3547bd43146: Download complete 2024-08-20T22:08:25.7651971Z 5645520256fa: Download complete 2024-08-20T22:08:25.8444849Z 829ed7d91e38: Verifying Checksum 2024-08-20T22:08:25.8445320Z 829ed7d91e38: Download complete 2024-08-20T22:08:27.1135007Z 385d0421d67c: Verifying Checksum 2024-08-20T22:08:27.1135709Z 385d0421d67c: Download complete 2024-08-20T22:08:27.1821635Z 7cbfce999d76: Download complete 2024-08-20T22:08:27.2490163Z 6658e558ebe4: Verifying Checksum 2024-08-20T22:08:27.2490634Z 6658e558ebe4: Download complete 2024-08-20T22:08:27.3154480Z 33444a5a7e01: Verifying Checksum 2024-08-20T22:08:27.3862360Z 7c47122f4e09: Verifying Checksum 2024-08-20T22:08:27.3863045Z 7c47122f4e09: Download complete 2024-08-20T22:08:27.4626258Z 1811ff4d3556: Verifying Checksum 2024-08-20T22:08:27.4626896Z 1811ff4d3556: Download complete 2024-08-20T22:08:33.4571585Z 7754ee068b33: Verifying Checksum 2024-08-20T22:08:33.4572195Z 7754ee068b33: Download complete 2024-08-20T22:08:33.5398912Z 0479fbac3361: Download complete 2024-08-20T22:08:33.6116843Z f1d6874dee9c: Verifying Checksum 2024-08-20T22:08:33.6117363Z f1d6874dee9c: Download complete 2024-08-20T22:08:34.0243963Z afc429e87b59: Verifying Checksum 2024-08-20T22:08:34.0244445Z afc429e87b59: Download complete 2024-08-20T22:08:34.0878145Z 019620471c25: Verifying Checksum 2024-08-20T22:08:34.0878743Z 019620471c25: Download complete 2024-08-20T22:08:34.1508566Z 27aeb1b2cd5e: Verifying Checksum 2024-08-20T22:08:34.1510676Z 27aeb1b2cd5e: Download complete 2024-08-20T22:08:34.3979749Z 829c0e19f4b7: Verifying Checksum 2024-08-20T22:08:34.3980224Z 829c0e19f4b7: Download complete 2024-08-20T22:08:34.4669227Z 560cd6122a38: Verifying Checksum 2024-08-20T22:08:34.4669821Z 560cd6122a38: Download complete 2024-08-20T22:08:34.5438444Z 5a4eb19d7219: Download complete 2024-08-20T22:08:34.6138724Z da6a838a66c7: Download complete 2024-08-20T22:08:37.7248195Z 7ed32bc8e469: Verifying Checksum 2024-08-20T22:08:37.7248689Z 7ed32bc8e469: Download complete 2024-08-20T22:08:37.7949200Z 4d4fa6802ab3: Verifying Checksum 2024-08-20T22:08:37.7949678Z 4d4fa6802ab3: Download complete 2024-08-20T22:08:37.8666128Z 0f085d4fa279: Download complete 2024-08-20T22:08:38.3461321Z 4a0412373305: Verifying Checksum 2024-08-20T22:08:38.3461881Z 4a0412373305: Download complete 2024-08-20T22:08:38.4161839Z 302c8c177944: Verifying Checksum 2024-08-20T22:08:38.4162404Z 302c8c177944: Download complete 2024-08-20T22:08:38.4925382Z 02970347dff6: Verifying Checksum 2024-08-20T22:08:38.4925971Z 02970347dff6: Download complete 2024-08-20T22:08:38.5676060Z 916c5e858016: Verifying Checksum 2024-08-20T22:08:38.5676615Z 916c5e858016: Download complete 2024-08-20T22:08:38.6331080Z c42b3602a1fb: Verifying Checksum 2024-08-20T22:08:38.6331617Z c42b3602a1fb: Download complete 2024-08-20T22:08:38.7909056Z a8c1e85b5e14: Pull complete 2024-08-20T22:08:38.9344296Z a41a8d1c11c8: Pull complete 2024-08-20T22:08:39.0500483Z 0c1227890755: Pull complete 2024-08-20T22:08:39.2539665Z d8d1234baab3: Pull complete 2024-08-20T22:08:45.2413566Z 6efb96fc87ea: Verifying Checksum 2024-08-20T22:08:45.2414082Z 6efb96fc87ea: Download complete 2024-08-20T22:08:45.3165460Z f957c1624481: Download complete 2024-08-20T22:08:45.4003634Z 768c31dca016: Verifying Checksum 2024-08-20T22:08:45.4789823Z 768c31dca016: Download complete 2024-08-20T22:08:45.4790427Z 8c32d0ecebbe: Verifying Checksum 2024-08-20T22:08:45.4790891Z 8c32d0ecebbe: Download complete 2024-08-20T22:08:45.4870916Z 4f4fb700ef54: Verifying Checksum 2024-08-20T22:08:45.4871455Z 4f4fb700ef54: Download complete 2024-08-20T22:08:45.5888502Z 78ee77eddef2: Download complete 2024-08-20T22:08:45.6692863Z 37513e9b43d7: Verifying Checksum 2024-08-20T22:08:45.6693464Z 37513e9b43d7: Download complete 2024-08-20T22:08:45.7981787Z a7a5392e2bf6: Verifying Checksum 2024-08-20T22:08:45.7982439Z a7a5392e2bf6: Download complete 2024-08-20T22:08:45.8690919Z b10de02255c0: Verifying Checksum 2024-08-20T22:08:45.8691414Z b10de02255c0: Download complete 2024-08-20T22:08:45.9862586Z e56eeff2db93: Verifying Checksum 2024-08-20T22:08:45.9863227Z e56eeff2db93: Download complete 2024-08-20T22:08:46.0562885Z 70ac3e38d0fd: Verifying Checksum 2024-08-20T22:08:46.0563492Z 70ac3e38d0fd: Download complete 2024-08-20T22:08:46.1290496Z eb09cbf3e416: Verifying Checksum 2024-08-20T22:08:46.1291043Z eb09cbf3e416: Download complete 2024-08-20T22:08:46.2753610Z ee1273153675: Verifying Checksum 2024-08-20T22:08:46.2754230Z ee1273153675: Download complete 2024-08-20T22:08:46.3345919Z e22979f9e0ee: Verifying Checksum 2024-08-20T22:08:46.3346425Z e22979f9e0ee: Download complete 2024-08-20T22:08:46.9344082Z 6c1f0aabce7c: Verifying Checksum 2024-08-20T22:08:46.9344578Z 6c1f0aabce7c: Download complete 2024-08-20T22:08:47.0124645Z 7ebe49c3a23e: Download complete 2024-08-20T22:09:16.4510956Z 5d115b29b5d5: Verifying Checksum 2024-08-20T22:09:16.4511431Z 5d115b29b5d5: Download complete 2024-08-20T22:09:16.5356269Z 7f2d7a04fbe6: Download complete 2024-08-20T22:09:16.5984772Z d79fd7d52698: Verifying Checksum 2024-08-20T22:09:16.5985424Z d79fd7d52698: Download complete 2024-08-20T22:09:17.9895231Z 9752d6fbbcab: Verifying Checksum 2024-08-20T22:09:17.9895864Z 9752d6fbbcab: Download complete 2024-08-20T22:09:18.0592762Z 83808b2eff4d: Download complete 2024-08-20T22:09:55.5649867Z cbb79a705353: Verifying Checksum 2024-08-20T22:09:55.8832016Z 11fad972ef6c: Verifying Checksum 2024-08-20T22:09:55.8832469Z 11fad972ef6c: Download complete 2024-08-20T22:10:11.8932917Z 7ed32bc8e469: Pull complete 2024-08-20T22:10:12.0874914Z ec1e7978c1fe: Pull complete 2024-08-20T22:10:12.3135872Z ac23c5fcc1ef: Pull complete 2024-08-20T22:10:20.9383266Z ca426ffc9d2b: Pull complete 2024-08-20T22:10:21.1378402Z 4373aa743caf: Pull complete 2024-08-20T22:10:21.3692981Z 75b2abb16c68: Pull complete 2024-08-20T22:10:21.5169364Z ef1e3ef1e85f: Pull complete 2024-08-20T22:10:24.1171729Z 0bf61114d37b: Pull complete 2024-08-20T22:10:24.3087503Z 2df849f72ba4: Pull complete 2024-08-20T22:10:24.5055705Z 34c47d63c5de: Pull complete 2024-08-20T22:10:24.5360520Z 1fef4bd19792: Pull complete 2024-08-20T22:11:22.3951797Z 6efb96fc87ea: Pull complete 2024-08-20T22:11:22.5451954Z b119c4fc6c58: Pull complete 2024-08-20T22:11:22.6820391Z 33b639226eee: Pull complete 2024-08-20T22:11:22.8135390Z d3547bd43146: Pull complete 2024-08-20T22:11:22.8884953Z 5645520256fa: Pull complete 2024-08-20T22:11:22.9864133Z 829ed7d91e38: Pull complete 2024-08-20T22:11:26.5956278Z 385d0421d67c: Pull complete 2024-08-20T22:11:26.7942569Z 7cbfce999d76: Pull complete 2024-08-20T22:11:27.0044555Z 6658e558ebe4: Pull complete 2024-08-20T22:11:27.2102876Z 33444a5a7e01: Pull complete 2024-08-20T22:11:27.4108269Z 7c47122f4e09: Pull complete 2024-08-20T22:11:27.6103785Z 1811ff4d3556: Pull complete 2024-08-20T22:11:38.2481952Z 7754ee068b33: Pull complete 2024-08-20T22:11:38.4055475Z 0479fbac3361: Pull complete 2024-08-20T22:11:38.5433776Z f1d6874dee9c: Pull complete 2024-08-20T22:11:39.5089744Z afc429e87b59: Pull complete 2024-08-20T22:11:39.6437043Z 019620471c25: Pull complete 2024-08-20T22:11:39.7306394Z 27aeb1b2cd5e: Pull complete 2024-08-20T22:11:40.1455994Z 829c0e19f4b7: Pull complete 2024-08-20T22:11:40.3184460Z 560cd6122a38: Pull complete 2024-08-20T22:11:40.5085606Z 5a4eb19d7219: Pull complete 2024-08-20T22:11:40.6252507Z da6a838a66c7: Pull complete 2024-08-20T22:12:19.5928520Z cbb79a705353: Pull complete 2024-08-20T22:12:19.7867619Z 4d4fa6802ab3: Pull complete 2024-08-20T22:12:19.8801997Z 0f085d4fa279: Pull complete 2024-08-20T22:12:22.2391730Z 4a0412373305: Pull complete 2024-08-20T22:12:22.2557696Z 302c8c177944: Pull complete 2024-08-20T22:12:22.2701117Z 02970347dff6: Pull complete 2024-08-20T22:12:22.2997903Z 916c5e858016: Pull complete 2024-08-20T22:12:22.3137474Z c42b3602a1fb: Pull complete 2024-08-20T22:13:43.9454484Z 11fad972ef6c: Pull complete 2024-08-20T22:13:44.1175419Z f957c1624481: Pull complete 2024-08-20T22:13:44.3155617Z 768c31dca016: Pull complete 2024-08-20T22:13:44.5905292Z 8c32d0ecebbe: Pull complete 2024-08-20T22:13:44.7257612Z 4f4fb700ef54: Pull complete 2024-08-20T22:13:44.8099735Z 78ee77eddef2: Pull complete 2024-08-20T22:13:44.9868051Z 37513e9b43d7: Pull complete 2024-08-20T22:13:45.5181661Z a7a5392e2bf6: Pull complete 2024-08-20T22:13:45.7011790Z b10de02255c0: Pull complete 2024-08-20T22:13:45.9094424Z e56eeff2db93: Pull complete 2024-08-20T22:13:46.0973196Z 70ac3e38d0fd: Pull complete 2024-08-20T22:13:46.2983210Z eb09cbf3e416: Pull complete 2024-08-20T22:13:47.7887142Z ee1273153675: Pull complete 2024-08-20T22:13:47.9746874Z e22979f9e0ee: Pull complete 2024-08-20T22:13:50.7149398Z 6c1f0aabce7c: Pull complete 2024-08-20T22:13:50.8123948Z 7ebe49c3a23e: Pull complete 2024-08-20T22:14:39.6703629Z 5d115b29b5d5: Pull complete 2024-08-20T22:14:39.8367822Z 7f2d7a04fbe6: Pull complete 2024-08-20T22:14:40.0280745Z d79fd7d52698: Pull complete 2024-08-20T22:14:41.9056474Z 9752d6fbbcab: Pull complete 2024-08-20T22:14:41.9972939Z 83808b2eff4d: Pull complete 2024-08-20T22:14:42.6116441Z Digest: sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T22:14:42.6417152Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:14:42.6620593Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:14:42.6719707Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:14:42.6720641Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:14:42.6733092Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:14:42.6733601Z env: 2024-08-20T22:14:42.6733884Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:42.6734233Z ##[endgroup] 2024-08-20T22:14:42.6930032Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-20T22:14:42.6930599Z with: 2024-08-20T22:14:42.6930881Z driver-version: 550.54.15 2024-08-20T22:14:42.6931220Z env: 2024-08-20T22:14:42.6931494Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:42.6931844Z ##[endgroup] 2024-08-20T22:14:42.7011219Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:14:42.7011819Z with: 2024-08-20T22:14:42.7012105Z timeout_minutes: 10 2024-08-20T22:14:42.7012443Z max_attempts: 3 2024-08-20T22:14:42.7046605Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-20T22:14:42.7081874Z retry_wait_seconds: 10 2024-08-20T22:14:42.7082248Z polling_interval_seconds: 1 2024-08-20T22:14:42.7082623Z warning_on_retry: true 2024-08-20T22:14:42.7082983Z continue_on_error: false 2024-08-20T22:14:42.7083326Z env: 2024-08-20T22:14:42.7083596Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:42.7083947Z DRIVER_VERSION: 550.54.15 2024-08-20T22:14:42.7084305Z ##[endgroup] 2024-08-20T22:14:43.1607675Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-20T22:14:43.1608312Z + pre_install_nvidia_driver_amzn2 2024-08-20T22:14:43.1609326Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-20T22:14:44.1755602Z No match for argument: nvidia-driver-latest-dkms 2024-08-20T22:14:44.1756161Z No packages marked for removal. 2024-08-20T22:14:44.1887620Z Dependencies resolved. 2024-08-20T22:14:44.1897956Z Nothing to do. 2024-08-20T22:14:44.1898521Z Complete! 2024-08-20T22:14:44.2645208Z + install_nvidia_driver_common 2024-08-20T22:14:44.2648856Z + echo 'Before installing NVIDIA driver' 2024-08-20T22:14:44.2649312Z + lspci 2024-08-20T22:14:44.2649613Z Before installing NVIDIA driver 2024-08-20T22:14:44.2758625Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:14:44.2759530Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:14:44.2760495Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:14:44.2761336Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:14:44.2762156Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:14:44.2762986Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:14:44.2763742Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:14:44.2764572Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:14:44.2765171Z + lsmod 2024-08-20T22:14:44.2811373Z Module Size Used by 2024-08-20T22:14:44.2812311Z ib_core 454656 0 2024-08-20T22:14:44.2813253Z xt_nat 16384 0 2024-08-20T22:14:44.2813986Z veth 36864 0 2024-08-20T22:14:44.2814725Z nvidia_modeset 1351680 0 2024-08-20T22:14:44.2815571Z video 65536 1 nvidia_modeset 2024-08-20T22:14:44.2816178Z wmi 36864 1 video 2024-08-20T22:14:44.2816594Z nvidia_uvm 4706304 0 2024-08-20T22:14:44.2817052Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-20T22:14:44.2817543Z drm 602112 1 nvidia 2024-08-20T22:14:44.2817992Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:14:44.2818524Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-20T22:14:44.2819051Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:14:44.2819509Z xt_conntrack 16384 1 2024-08-20T22:14:44.2820008Z nft_chain_nat 16384 3 2024-08-20T22:14:44.2820516Z xt_MASQUERADE 20480 1 2024-08-20T22:14:44.2821032Z nf_nat 57344 3 xt_nat,nft_chain_nat,xt_MASQUERADE 2024-08-20T22:14:44.2821701Z nf_conntrack_netlink 57344 0 2024-08-20T22:14:44.2822443Z nf_conntrack 184320 5 xt_conntrack,nf_nat,xt_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:14:44.2823343Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:14:44.2824077Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:14:44.2824517Z xfrm_user 57344 1 2024-08-20T22:14:44.2824922Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:14:44.2825346Z xt_addrtype 16384 2 2024-08-20T22:14:44.2825732Z nft_compat 20480 4 2024-08-20T22:14:44.2826197Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:14:44.2826863Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:14:44.2827418Z br_netfilter 36864 0 2024-08-20T22:14:44.2827834Z bridge 307200 1 br_netfilter 2024-08-20T22:14:44.2828286Z stp 16384 1 bridge 2024-08-20T22:14:44.2828719Z llc 16384 2 bridge,stp 2024-08-20T22:14:44.2829138Z overlay 167936 0 2024-08-20T22:14:44.2829511Z tls 114688 0 2024-08-20T22:14:44.2829882Z nls_ascii 16384 1 2024-08-20T22:14:44.2830253Z nls_cp437 20480 1 2024-08-20T22:14:44.2830624Z vfat 24576 1 2024-08-20T22:14:44.2830998Z fat 86016 1 vfat 2024-08-20T22:14:44.2831397Z sunrpc 692224 1 2024-08-20T22:14:44.2831779Z ghash_clmulni_intel 16384 0 2024-08-20T22:14:44.2832153Z ena 167936 0 2024-08-20T22:14:44.2832556Z ptp 36864 1 ena 2024-08-20T22:14:44.2833023Z pps_core 24576 1 ptp 2024-08-20T22:14:44.2833430Z aesni_intel 393216 0 2024-08-20T22:14:44.2833921Z i8042 45056 0 2024-08-20T22:14:44.2834313Z serio 28672 3 i8042 2024-08-20T22:14:44.2834881Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:14:44.2835580Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:14:44.2836098Z button 24576 0 2024-08-20T22:14:44.2836480Z sch_fq_codel 20480 17 2024-08-20T22:14:44.2836859Z dm_mod 188416 0 2024-08-20T22:14:44.2837240Z fuse 163840 1 2024-08-20T22:14:44.2837619Z dax 45056 1 dm_mod 2024-08-20T22:14:44.2838025Z configfs 57344 1 2024-08-20T22:14:44.2838393Z loop 36864 0 2024-08-20T22:14:44.2838765Z dmi_sysfs 20480 0 2024-08-20T22:14:44.2839143Z crc32_pclmul 16384 0 2024-08-20T22:14:44.2839517Z crc32c_intel 24576 0 2024-08-20T22:14:44.2839887Z efivarfs 24576 1 2024-08-20T22:14:44.2840257Z + modinfo nvidia 2024-08-20T22:14:44.2840991Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:14:44.2841855Z alias: char-major-195-* 2024-08-20T22:14:44.2842254Z version: 550.54.15 2024-08-20T22:14:44.2842604Z supported: external 2024-08-20T22:14:44.2842952Z license: NVIDIA 2024-08-20T22:14:44.2843339Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:14:44.2843828Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:14:44.2844306Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:14:44.2844788Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:14:44.2845300Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:14:44.2845794Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:14:44.2846314Z depends: i2c-core,drm 2024-08-20T22:14:44.2846684Z retpoline: Y 2024-08-20T22:14:44.2847294Z name: nvidia 2024-08-20T22:14:44.2847957Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:14:44.2848675Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:14:44.2849337Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:14:44.2849945Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:14:44.2850394Z parm: NVreg_RmLogonRC:int 2024-08-20T22:14:44.2850838Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:14:44.2851307Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:14:44.2851912Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:14:44.2852362Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:14:44.2852903Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:14:44.2853477Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:14:44.2853962Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:14:44.2854402Z parm: NVreg_EnableMSI:int 2024-08-20T22:14:44.2854828Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:14:44.2855301Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:14:44.2855845Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:14:44.2856445Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:14:44.2857017Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:14:44.2857639Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:44.2858245Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:14:44.2858876Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:44.2859483Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:14:44.2859986Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:14:44.2860526Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:14:44.2861080Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:14:44.2861730Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:14:44.2862207Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:14:44.2862694Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:14:44.2863180Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:14:44.2863642Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:14:44.2864156Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:14:44.2864685Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:14:44.2865172Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:14:44.2865705Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:14:44.2866233Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:14:44.2866733Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:14:44.2867244Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:14:44.2867742Z parm: NVreg_RmMsg:charp 2024-08-20T22:14:44.2868157Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:14:44.2868637Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:14:44.2869122Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:14:44.2869585Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:14:44.2870210Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:14:44.2870700Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:14:44.2871165Z parm: rm_firmware_active:charp 2024-08-20T22:14:44.2871590Z + HAS_NVIDIA_DRIVER=0 2024-08-20T22:14:44.2871977Z ++ command -v nvidia-smi 2024-08-20T22:14:44.2872387Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-20T22:14:44.2872770Z + set +e 2024-08-20T22:14:44.2873281Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-20T22:14:44.3118282Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-20T22:14:44.3119577Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:14:44.3120287Z + '[' 0 -ne 0 ']' 2024-08-20T22:14:44.3120668Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-20T22:14:44.3121071Z + HAS_NVIDIA_DRIVER=1 2024-08-20T22:14:44.3121826Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-20T22:14:44.3122546Z + set -e 2024-08-20T22:14:44.3122852Z + '[' 1 -eq 0 ']' 2024-08-20T22:14:44.3123471Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-20T22:14:44.3124191Z + post_install_nvidia_driver_common 2024-08-20T22:14:44.3126145Z + sudo modprobe nvidia 2024-08-20T22:14:44.4441670Z + echo 'After installing NVIDIA driver' 2024-08-20T22:14:44.4442159Z + lspci 2024-08-20T22:14:44.4442466Z After installing NVIDIA driver 2024-08-20T22:14:44.4546368Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:14:44.4547614Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:14:44.4548610Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:14:44.4549511Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:14:44.4550353Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:14:44.4551170Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:14:44.4551939Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:14:44.4552774Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:14:44.4553386Z + lsmod 2024-08-20T22:14:44.4580489Z Module Size Used by 2024-08-20T22:14:44.4581021Z ib_core 454656 0 2024-08-20T22:14:44.4581681Z xt_nat 16384 0 2024-08-20T22:14:44.4582189Z veth 36864 0 2024-08-20T22:14:44.4582708Z nvidia_modeset 1351680 0 2024-08-20T22:14:44.4583158Z video 65536 1 nvidia_modeset 2024-08-20T22:14:44.4583605Z wmi 36864 1 video 2024-08-20T22:14:44.4584003Z nvidia_uvm 4706304 0 2024-08-20T22:14:44.4584467Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-20T22:14:44.4584952Z drm 602112 1 nvidia 2024-08-20T22:14:44.4585401Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:14:44.4585935Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-20T22:14:44.4586457Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:14:44.4586880Z xt_conntrack 16384 1 2024-08-20T22:14:44.4587273Z nft_chain_nat 16384 3 2024-08-20T22:14:44.4587661Z xt_MASQUERADE 20480 1 2024-08-20T22:14:44.4588178Z nf_nat 57344 3 xt_nat,nft_chain_nat,xt_MASQUERADE 2024-08-20T22:14:44.4588817Z nf_conntrack_netlink 57344 0 2024-08-20T22:14:44.4589470Z nf_conntrack 184320 5 xt_conntrack,nf_nat,xt_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:14:44.4590193Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:14:44.4590681Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:14:44.4591122Z xfrm_user 57344 1 2024-08-20T22:14:44.4591530Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:14:44.4591990Z xt_addrtype 16384 2 2024-08-20T22:14:44.4592379Z nft_compat 20480 4 2024-08-20T22:14:44.4592841Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:14:44.4593696Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:14:44.4594268Z br_netfilter 36864 0 2024-08-20T22:14:44.4594692Z bridge 307200 1 br_netfilter 2024-08-20T22:14:44.4595140Z stp 16384 1 bridge 2024-08-20T22:14:44.4595573Z llc 16384 2 bridge,stp 2024-08-20T22:14:44.4596009Z overlay 167936 0 2024-08-20T22:14:44.4596387Z tls 114688 0 2024-08-20T22:14:44.4596758Z nls_ascii 16384 1 2024-08-20T22:14:44.4597140Z nls_cp437 20480 1 2024-08-20T22:14:44.4597518Z vfat 24576 1 2024-08-20T22:14:44.4597890Z fat 86016 1 vfat 2024-08-20T22:14:44.4598292Z sunrpc 692224 1 2024-08-20T22:14:44.4598681Z ghash_clmulni_intel 16384 0 2024-08-20T22:14:44.4599069Z ena 167936 0 2024-08-20T22:14:44.4599445Z ptp 36864 1 ena 2024-08-20T22:14:44.4599907Z pps_core 24576 1 ptp 2024-08-20T22:14:44.4600432Z aesni_intel 393216 0 2024-08-20T22:14:44.4600924Z i8042 45056 0 2024-08-20T22:14:44.4601406Z serio 28672 3 i8042 2024-08-20T22:14:44.4601946Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:14:44.4602468Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:14:44.4603108Z button 24576 0 2024-08-20T22:14:44.4603479Z sch_fq_codel 20480 17 2024-08-20T22:14:44.4603851Z dm_mod 188416 0 2024-08-20T22:14:44.4604214Z fuse 163840 1 2024-08-20T22:14:44.4604580Z dax 45056 1 dm_mod 2024-08-20T22:14:44.4604978Z configfs 57344 1 2024-08-20T22:14:44.4605342Z loop 36864 0 2024-08-20T22:14:44.4605705Z dmi_sysfs 20480 0 2024-08-20T22:14:44.4606070Z crc32_pclmul 16384 0 2024-08-20T22:14:44.4606435Z crc32c_intel 24576 0 2024-08-20T22:14:44.4606806Z efivarfs 24576 1 2024-08-20T22:14:44.4607160Z + modinfo nvidia 2024-08-20T22:14:44.4607860Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:14:44.4608554Z alias: char-major-195-* 2024-08-20T22:14:44.4608938Z version: 550.54.15 2024-08-20T22:14:44.4609281Z supported: external 2024-08-20T22:14:44.4609619Z license: NVIDIA 2024-08-20T22:14:44.4609996Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:14:44.4610478Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:14:44.4610934Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:14:44.4611387Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:14:44.4611884Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:14:44.4612368Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:14:44.4612851Z depends: i2c-core,drm 2024-08-20T22:14:44.4613214Z retpoline: Y 2024-08-20T22:14:44.4613530Z name: nvidia 2024-08-20T22:14:44.4614135Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:14:44.4614836Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:14:44.4615484Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:14:44.4616077Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:14:44.4616526Z parm: NVreg_RmLogonRC:int 2024-08-20T22:14:44.4616954Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:14:44.4617418Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:14:44.4617856Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:14:44.4618296Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:14:44.4618824Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:14:44.4619385Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:14:44.4619861Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:14:44.4620404Z parm: NVreg_EnableMSI:int 2024-08-20T22:14:44.4620820Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:14:44.4621284Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:14:44.4621964Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:14:44.4622544Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:14:44.4623104Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:14:44.4623712Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:44.4624305Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:14:44.4624910Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:44.4625501Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:14:44.4626000Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:14:44.4626536Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:14:44.4627079Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:14:44.4627578Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:14:44.4628042Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:14:44.4628523Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:14:44.4628989Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:14:44.4629435Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:14:44.4630040Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:14:44.4630565Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:14:44.4631038Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:14:44.4631526Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:14:44.4632032Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:14:44.4632516Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:14:44.4633010Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:14:44.4633497Z parm: NVreg_RmMsg:charp 2024-08-20T22:14:44.4633909Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:14:44.4634377Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:14:44.4634846Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:14:44.4635303Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:14:44.4635839Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:14:44.4636309Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:14:44.4636771Z parm: rm_firmware_active:charp 2024-08-20T22:14:44.4637172Z + set +e 2024-08-20T22:14:44.4637485Z + nvidia-smi 2024-08-20T22:14:44.5158408Z Tue Aug 20 22:14:44 2024 2024-08-20T22:14:44.5159086Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:44.5159927Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:14:44.5160742Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:14:44.5161571Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:14:44.5162496Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:14:44.5163203Z | | | MIG M. | 2024-08-20T22:14:44.5163760Z |=========================================+========================+======================| 2024-08-20T22:14:44.5394494Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:14:44.5395236Z | 0% 26C P8 22W / 300W | 0MiB / 23028MiB | 0% Default | 2024-08-20T22:14:44.5395941Z | | | N/A | 2024-08-20T22:14:44.5396651Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:14:44.5399537Z 2024-08-20T22:14:44.5400500Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:44.5401141Z | Processes: | 2024-08-20T22:14:44.5401864Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:14:44.5402577Z | ID ID Usage | 2024-08-20T22:14:44.5403159Z |=========================================================================================| 2024-08-20T22:14:44.5405178Z | No running processes found | 2024-08-20T22:14:44.5405984Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:44.8388715Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-20T22:14:44.8574112Z NVIDIA A10G 2024-08-20T22:14:44.8634872Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:14:44.8635322Z + '[' 0 -eq 0 ']' 2024-08-20T22:14:44.8635747Z + echo 'INFO: Ignoring allowed status 0' 2024-08-20T22:14:44.8636193Z + set -e 2024-08-20T22:14:44.8636501Z INFO: Ignoring allowed status 0 2024-08-20T22:14:44.8644364Z == Installing nvidia container toolkit for amzn2023 == 2024-08-20T22:14:44.8650440Z + sudo yum install -y yum-utils 2024-08-20T22:14:45.4662092Z Last metadata expiration check: 0:34:13 ago on Tue Aug 20 21:40:32 2024. 2024-08-20T22:14:45.4880600Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-20T22:14:45.5183999Z Dependencies resolved. 2024-08-20T22:14:45.5306732Z Nothing to do. 2024-08-20T22:14:45.5307058Z Complete! 2024-08-20T22:14:45.5965686Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-20T22:14:45.5966835Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:14:45.5968198Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:14:45.8704533Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:14:45.9426820Z + sudo yum install -y nvidia-docker2 2024-08-20T22:14:46.4486388Z nvidia-container-toolkit 11 kB/s | 833 B 00:00 2024-08-20T22:14:46.4705034Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-20T22:14:46.5001036Z Dependencies resolved. 2024-08-20T22:14:46.5123913Z Nothing to do. 2024-08-20T22:14:46.5124477Z Complete! 2024-08-20T22:14:46.5932001Z + sudo systemctl restart docker 2024-08-20T22:15:18.7724097Z nvidia-persistenced failed to initialize. Check syslog for more details. 2024-08-20T22:15:18.7964154Z Tue Aug 20 22:15:18 2024 2024-08-20T22:15:18.7965174Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:15:18.7966032Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:15:18.7967142Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:15:18.7968115Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:15:18.7969040Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:15:18.7969748Z | | | MIG M. | 2024-08-20T22:15:18.7970320Z |=========================================+========================+======================| 2024-08-20T22:15:18.8156605Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:15:18.8157421Z | 0% 26C P8 22W / 300W | 0MiB / 23028MiB | 0% Default | 2024-08-20T22:15:18.8158072Z | | | N/A | 2024-08-20T22:15:18.8159132Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:15:18.8161992Z 2024-08-20T22:15:18.8162689Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:15:18.8163343Z | Processes: | 2024-08-20T22:15:18.8164087Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:15:18.8164812Z | ID ID Usage | 2024-08-20T22:15:18.8165386Z |=========================================================================================| 2024-08-20T22:15:18.8168637Z | No running processes found | 2024-08-20T22:15:18.8169429Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:15:19.8337171Z Command completed after 1 attempt(s). 2024-08-20T22:15:19.8416591Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:15:19.8417374Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:15:19.8418076Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T22:15:19.8418965Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:19.8431717Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:19.8432219Z env: 2024-08-20T22:15:19.8432493Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:19.8432950Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:19.8433438Z ##[endgroup] 2024-08-20T22:15:20.1902385Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:15:20.2116111Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-08-20T22:15:20.2121643Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-08-20T22:15:20.3395718Z Prepare all required actions 2024-08-20T22:15:20.3396221Z Getting action download info 2024-08-20T22:15:20.4602604Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T22:15:20.6354238Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T22:15:20.7674384Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T22:15:20.7674881Z with: 2024-08-20T22:15:20.7675219Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:20.7675693Z s3-bucket: gha-artifacts 2024-08-20T22:15:20.7676039Z env: 2024-08-20T22:15:20.7676326Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:20.7676780Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:20.7677276Z ##[endgroup] 2024-08-20T22:15:20.7711627Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:15:20.7712083Z with: 2024-08-20T22:15:20.7712467Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:20.7712931Z s3-bucket: gha-artifacts 2024-08-20T22:15:20.7713293Z region: us-east-1 2024-08-20T22:15:20.7713604Z env: 2024-08-20T22:15:20.7713890Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:20.7714346Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:20.7714858Z ##[endgroup] 2024-08-20T22:15:21.2710544Z (node:375251) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:15:21.2711516Z 2024-08-20T22:15:21.2711859Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:15:21.2712609Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:15:21.2713563Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:15:21.3428662Z Found 1 objects with prefix pytorch/pytorch/10479310721/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-08-20T22:15:21.3429945Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:15:28.8748438Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:15:28.8755632Z Artifact download has finished successfully 2024-08-20T22:15:28.9098576Z ##[group]Run unzip -o artifacts.zip 2024-08-20T22:15:28.9099048Z unzip -o artifacts.zip 2024-08-20T22:15:28.9110557Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:28.9111065Z env: 2024-08-20T22:15:28.9111352Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:28.9111818Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:28.9112311Z ##[endgroup] 2024-08-20T22:15:28.9181430Z Archive: artifacts.zip 2024-08-20T22:15:28.9183219Z creating: dist/ 2024-08-20T22:15:31.0465811Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:15:31.0466498Z creating: build/custom_test_artifacts/ 2024-08-20T22:15:31.0467108Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T22:15:31.0467872Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T22:15:31.0468749Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:31.0475540Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:31.0476821Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:31.0477800Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:31.0478841Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:31.0479855Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:31.0481000Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:31.0482532Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:31.0483598Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:31.0484636Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:31.0485995Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:31.0488198Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:31.0489672Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:31.0490876Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:31.0492359Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:31.0493853Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:31.0494940Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:31.0495989Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:31.0535857Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:31.0575720Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:31.0577228Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:31.0622663Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:31.0624149Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:31.0625647Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:31.0627175Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:31.0628662Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:31.0630108Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:31.0631563Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:31.0633014Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:31.0634420Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:31.0635752Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:31.0637293Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:31.0638558Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:31.0639831Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:31.0641059Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:31.0642311Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:31.0698313Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:31.0758567Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:31.0759806Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:31.0760841Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:31.0761750Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:31.0762697Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:31.0763666Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T22:15:31.0764732Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T22:15:31.0765936Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T22:15:31.0767087Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T22:15:31.0768171Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T22:15:31.0769301Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T22:15:31.0770419Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T22:15:31.0771549Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T22:15:31.0772664Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T22:15:31.0773770Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T22:15:31.0790947Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T22:15:31.0936549Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T22:15:31.0938625Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T22:15:31.0939942Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T22:15:31.0941279Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T22:15:31.0942491Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T22:15:31.0943622Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T22:15:31.0944794Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T22:15:31.0945979Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T22:15:31.0947448Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T22:15:31.0948789Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T22:15:31.0950130Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T22:15:31.0965448Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T22:15:31.1049711Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T22:15:31.1050957Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:31.1052077Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:31.1053259Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T22:15:31.1054223Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T22:15:31.1055158Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:31.1056105Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-20T22:15:31.1058192Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T22:15:31.1059201Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T22:15:31.1060340Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T22:15:31.1180072Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T22:15:31.1243306Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T22:15:31.1244075Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T22:15:31.1244823Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T22:15:31.1245675Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:31.1252414Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:31.1253393Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:31.1254366Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:31.1255394Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:31.1256395Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:31.1257532Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:31.1258913Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:31.1259953Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:31.1260968Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:31.1262470Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:31.1263830Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:31.1265683Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:31.1266872Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:31.1268507Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:31.1269972Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:31.1271057Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:31.1272088Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:31.1312628Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:31.1352871Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:31.1354357Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:31.1399749Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:31.1401205Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:31.1402883Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:31.1404413Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:31.1405895Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:31.1407332Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:31.1408773Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:31.1410214Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:31.1411605Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:31.1412939Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:31.1414212Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:31.1415476Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:31.1416735Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:31.1417966Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:31.1419214Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:31.1475344Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:31.1535374Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:31.1536883Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:31.1537914Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:31.1538815Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:31.1539760Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:31.1540743Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T22:15:31.1541937Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T22:15:31.1543170Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T22:15:31.1544359Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T22:15:31.1545474Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T22:15:31.1546612Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T22:15:31.1548093Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T22:15:31.1549310Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T22:15:31.1550448Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T22:15:31.1551585Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T22:15:31.1569556Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T22:15:31.1634321Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T22:15:31.1635553Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:31.1636675Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:31.1637687Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T22:15:31.1638623Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T22:15:31.1639542Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:31.1640470Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-20T22:15:31.1642795Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T22:15:31.1643949Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T22:15:31.1644833Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T22:15:31.1694949Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T22:15:31.1695737Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T22:15:31.1696550Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T22:15:31.1697471Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:31.1703890Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:31.1704923Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:31.1705951Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:31.1707144Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:31.1708207Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:31.1709407Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:31.1710612Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:31.1711704Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:31.1712784Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:31.1714485Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:31.1715784Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:31.1717373Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:31.1718638Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:31.1720130Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:31.1721945Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:31.1723088Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:31.1724186Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:31.1764297Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:31.1803852Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:31.1805619Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:31.1851570Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:31.1853124Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:31.1854679Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:31.1856254Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:31.1857781Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:31.1859281Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:31.1860781Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:31.1862352Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:31.1863802Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:31.1865192Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:31.1866529Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:31.1867841Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:31.1869181Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:31.1870466Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:31.1871780Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:31.1927587Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:31.1987445Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:31.1988731Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:31.1989815Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:31.1990782Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:31.1991792Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:31.1992845Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T22:15:31.1994235Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T22:15:31.1995544Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T22:15:31.1996808Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T22:15:31.1997994Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T22:15:31.1999215Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T22:15:31.2000564Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T22:15:31.2001803Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T22:15:31.2003034Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T22:15:31.2004261Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T22:15:31.2005564Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T22:15:31.2130384Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T22:15:31.2131611Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T22:15:31.2132844Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T22:15:31.2134218Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T22:15:31.2135532Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T22:15:31.2136792Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T22:15:31.2138084Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T22:15:31.2139374Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T22:15:31.2140669Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T22:15:31.2142065Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T22:15:31.2143353Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T22:15:31.2160673Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T22:15:31.2216074Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T22:15:31.2217428Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:31.2218620Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:31.2219695Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T22:15:31.2220705Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T22:15:31.2221750Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:31.2222748Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-20T22:15:31.2224447Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T22:15:31.2225667Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T22:15:31.2227454Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T22:15:31.2330294Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T22:15:31.2373810Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T22:15:31.2374481Z creating: build/lib/ 2024-08-20T22:15:31.2463002Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T22:15:31.2920674Z inflating: build/lib/libprotobuf.a 2024-08-20T22:15:31.2929738Z inflating: build/lib/libpthreadpool.a 2024-08-20T22:15:31.2938378Z inflating: build/lib/libcpuinfo.a 2024-08-20T22:15:31.2946688Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T22:15:31.2947721Z inflating: build/lib/libclog.a 2024-08-20T22:15:31.2966339Z inflating: build/lib/libnnpack.a 2024-08-20T22:15:31.2968902Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T22:15:31.3034110Z inflating: build/lib/libgtest.a 2024-08-20T22:15:31.3110002Z inflating: build/lib/libbenchmark.a 2024-08-20T22:15:31.3173868Z inflating: build/lib/libasmjit.a 2024-08-20T22:15:31.3181854Z inflating: build/lib/libittnotify.a 2024-08-20T22:15:31.3210142Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T22:15:31.3336769Z inflating: build/lib/libgloo.a 2024-08-20T22:15:31.3358145Z inflating: build/lib/libfmt.a 2024-08-20T22:15:31.3454092Z inflating: build/lib/libc10.so 2024-08-20T22:15:31.3455330Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-20T22:15:31.3457217Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T22:15:31.3961496Z inflating: build/lib/libprotoc.a 2024-08-20T22:15:31.3981100Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T22:15:31.3999449Z inflating: build/lib/libgmock.a 2024-08-20T22:15:31.4000243Z inflating: build/lib/libgtest_main.a 2024-08-20T22:15:31.4001007Z inflating: build/lib/libbenchmark_main.a 2024-08-20T22:15:31.4391948Z inflating: build/lib/libgloo_cuda.a 2024-08-20T22:15:32.4589229Z inflating: build/lib/libdnnl.a 2024-08-20T22:15:32.5163389Z inflating: build/lib/libtensorpipe.a 2024-08-20T22:15:32.5220869Z inflating: build/lib/libc10_cuda.so 2024-08-20T22:15:32.5222100Z inflating: build/lib/libgmock_main.a 2024-08-20T22:15:32.6481452Z inflating: build/lib/libfbgemm.a 2024-08-20T22:15:32.6736672Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-20T22:15:32.7244292Z inflating: build/lib/libkineto.a 2024-08-20T22:15:32.7428708Z inflating: build/lib/libXNNPACK.a 2024-08-20T22:15:32.7471345Z inflating: build/lib/libonnx_proto.a 2024-08-20T22:15:32.8165908Z inflating: build/lib/libonnx.a 2024-08-20T22:15:35.2650076Z inflating: build/lib/libtorch_cpu.so 2024-08-20T22:15:35.2654864Z inflating: build/lib/libunbox_lib.a 2024-08-20T22:15:35.2659128Z inflating: build/lib/libshm.so 2024-08-20T22:15:37.3340581Z inflating: build/lib/libtorch_cuda.so 2024-08-20T22:15:37.3341972Z inflating: build/lib/libtorch.so 2024-08-20T22:15:37.3345019Z inflating: build/lib/libc10d_cuda_test.so 2024-08-20T22:15:38.1522466Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-20T22:15:38.3516669Z inflating: build/lib/libtorch_python.so 2024-08-20T22:15:38.3590060Z inflating: build/lib/libtorchbind_test.so 2024-08-20T22:15:38.3610091Z inflating: build/lib/libjitbackend_test.so 2024-08-20T22:15:38.3635758Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T22:15:38.3662142Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T22:15:38.3696837Z inflating: build/lib/libnnapi_backend.so 2024-08-20T22:15:38.3697404Z creating: build/bin/ 2024-08-20T22:15:38.3748749Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T22:15:38.3799826Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T22:15:38.3852194Z inflating: build/bin/c10_Device_test 2024-08-20T22:15:38.3911402Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T22:15:38.3965949Z inflating: build/bin/c10_Scalar_test 2024-08-20T22:15:38.4015041Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T22:15:38.4066810Z inflating: build/bin/c10_SymInt_test 2024-08-20T22:15:38.4122262Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T22:15:38.4178628Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T22:15:38.4234536Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T22:15:38.4305850Z inflating: build/bin/c10_cow_test 2024-08-20T22:15:38.4359380Z inflating: build/bin/c10_Bitset_test 2024-08-20T22:15:38.4408768Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T22:15:38.4459896Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T22:15:38.4510728Z inflating: build/bin/c10_Half_test 2024-08-20T22:15:38.4567637Z inflating: build/bin/c10_LeftRight_test 2024-08-20T22:15:38.4622902Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T22:15:38.4673619Z inflating: build/bin/c10_Synchronized_test 2024-08-20T22:15:38.4729591Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T22:15:38.4782503Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T22:15:38.4833162Z inflating: build/bin/c10_TypeList_test 2024-08-20T22:15:38.4882775Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T22:15:38.4934681Z inflating: build/bin/c10_accumulate_test 2024-08-20T22:15:38.4991787Z inflating: build/bin/c10_bfloat16_test 2024-08-20T22:15:38.5042351Z inflating: build/bin/c10_bit_cast_test 2024-08-20T22:15:38.5099420Z inflating: build/bin/c10_complex_math_test 2024-08-20T22:15:38.5155760Z inflating: build/bin/c10_complex_test 2024-08-20T22:15:38.5209403Z inflating: build/bin/c10_exception_test 2024-08-20T22:15:38.5259702Z inflating: build/bin/c10_flags_test 2024-08-20T22:15:38.5310103Z inflating: build/bin/c10_generic_math_test 2024-08-20T22:15:38.5475328Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T22:15:38.5526670Z inflating: build/bin/c10_irange_test 2024-08-20T22:15:38.5581134Z inflating: build/bin/c10_lazy_test 2024-08-20T22:15:38.5638149Z inflating: build/bin/c10_logging_test 2024-08-20T22:15:38.5713740Z inflating: build/bin/c10_optional_test 2024-08-20T22:15:38.5777356Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T22:15:38.5830731Z inflating: build/bin/c10_registry_test 2024-08-20T22:15:38.5982735Z inflating: build/bin/c10_small_vector_test 2024-08-20T22:15:38.6034540Z inflating: build/bin/c10_ssize_test 2024-08-20T22:15:38.6087362Z inflating: build/bin/c10_string_util_test 2024-08-20T22:15:38.6138665Z inflating: build/bin/c10_tempfile_test 2024-08-20T22:15:38.6197281Z inflating: build/bin/c10_string_view_test 2024-08-20T22:15:38.6246021Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T22:15:38.6302771Z inflating: build/bin/c10_typeid_test 2024-08-20T22:15:38.6751308Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T22:15:38.7199613Z inflating: build/bin/protoc 2024-08-20T22:15:38.7253220Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-20T22:15:38.7306597Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-20T22:15:38.7359828Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-20T22:15:38.7412039Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-20T22:15:38.7465645Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-20T22:15:38.7518917Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-20T22:15:38.7568346Z inflating: build/bin/c10_cuda_CUDATest 2024-08-20T22:15:38.7621906Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-20T22:15:38.7953361Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T22:15:38.8298890Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T22:15:38.8659395Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T22:15:38.8712400Z inflating: build/bin/BackoffTest 2024-08-20T22:15:38.8765814Z inflating: build/bin/FileStoreTest 2024-08-20T22:15:38.8822762Z inflating: build/bin/TCPStoreTest 2024-08-20T22:15:38.8875886Z inflating: build/bin/HashStoreTest 2024-08-20T22:15:38.8889531Z inflating: build/bin/ProcessGroupMPITest 2024-08-20T22:15:38.8943777Z inflating: build/bin/test_edge_op_registration 2024-08-20T22:15:38.8948461Z inflating: build/bin/torch_shm_manager 2024-08-20T22:15:38.8951582Z inflating: build/bin/example_allreduce 2024-08-20T22:15:38.9007306Z inflating: build/bin/test_dist_autograd 2024-08-20T22:15:38.9077291Z inflating: build/bin/test_cpp_rpc 2024-08-20T22:15:38.9080037Z inflating: build/bin/parallel_benchmark 2024-08-20T22:15:38.9147966Z inflating: build/bin/test_mobile_nnc 2024-08-20T22:15:38.9156739Z inflating: build/bin/aot_model_compiler_test 2024-08-20T22:15:38.9503600Z inflating: build/bin/test_lazy 2024-08-20T22:15:39.0688944Z inflating: build/bin/test_api 2024-08-20T22:15:39.0762996Z inflating: build/bin/Dict_test 2024-08-20T22:15:39.0816109Z inflating: build/bin/Dimname_test 2024-08-20T22:15:39.0880602Z inflating: build/bin/MaybeOwned_test 2024-08-20T22:15:39.0938684Z inflating: build/bin/NamedTensor_test 2024-08-20T22:15:39.0998042Z inflating: build/bin/apply_utils_test 2024-08-20T22:15:39.1057982Z inflating: build/bin/atest 2024-08-20T22:15:39.1121586Z inflating: build/bin/basic 2024-08-20T22:15:39.1176597Z inflating: build/bin/broadcast_test 2024-08-20T22:15:39.1228417Z inflating: build/bin/cpu_allocator_test 2024-08-20T22:15:39.1286852Z inflating: build/bin/cpu_generator_test 2024-08-20T22:15:39.1340614Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T22:15:39.1433899Z inflating: build/bin/cpu_rng_test 2024-08-20T22:15:39.1484545Z inflating: build/bin/dispatch_key_set_test 2024-08-20T22:15:39.1535109Z inflating: build/bin/dlconvertor_test 2024-08-20T22:15:39.1593984Z inflating: build/bin/extension_backend_test 2024-08-20T22:15:39.1649904Z inflating: build/bin/half_test 2024-08-20T22:15:39.1746250Z inflating: build/bin/ivalue_test 2024-08-20T22:15:39.1796765Z inflating: build/bin/lazy_tensor_test 2024-08-20T22:15:39.1851769Z inflating: build/bin/math_kernel_test 2024-08-20T22:15:39.1906340Z inflating: build/bin/memory_format_test 2024-08-20T22:15:39.1960397Z inflating: build/bin/memory_overlapping_test 2024-08-20T22:15:39.2013515Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T22:15:39.2070434Z inflating: build/bin/native_test 2024-08-20T22:15:39.2121863Z inflating: build/bin/operator_name_test 2024-08-20T22:15:39.2174818Z inflating: build/bin/operators_test 2024-08-20T22:15:39.2226801Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T22:15:39.2294502Z inflating: build/bin/pow_test 2024-08-20T22:15:39.2351455Z inflating: build/bin/quantized_test 2024-08-20T22:15:39.2401938Z inflating: build/bin/reduce_ops_test 2024-08-20T22:15:39.2453641Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T22:15:39.2510550Z inflating: build/bin/scalar_tensor_test 2024-08-20T22:15:39.2569809Z inflating: build/bin/scalar_test 2024-08-20T22:15:39.2622161Z inflating: build/bin/StorageUtils_test 2024-08-20T22:15:39.2675345Z inflating: build/bin/stride_properties_test 2024-08-20T22:15:39.2754780Z inflating: build/bin/tensor_iterator_test 2024-08-20T22:15:39.2809134Z inflating: build/bin/test_parallel 2024-08-20T22:15:39.2813015Z inflating: build/bin/thread_init_test 2024-08-20T22:15:39.2868757Z inflating: build/bin/type_ptr_test 2024-08-20T22:15:39.2930025Z inflating: build/bin/type_test 2024-08-20T22:15:39.2982124Z inflating: build/bin/undefined_tensor_test 2024-08-20T22:15:39.2983459Z inflating: build/bin/verify_api_visibility 2024-08-20T22:15:39.3053965Z inflating: build/bin/legacy_vmap_test 2024-08-20T22:15:39.3105946Z inflating: build/bin/weakref_test 2024-08-20T22:15:39.3157850Z inflating: build/bin/wrapdim_test 2024-08-20T22:15:39.3210333Z inflating: build/bin/xla_tensor_test 2024-08-20T22:15:39.3270532Z inflating: build/bin/IListRef_test 2024-08-20T22:15:39.3377815Z inflating: build/bin/List_test 2024-08-20T22:15:39.3444926Z inflating: build/bin/KernelFunction_test 2024-08-20T22:15:39.3567440Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T22:15:39.3664870Z inflating: build/bin/kernel_function_test 2024-08-20T22:15:39.3794337Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T22:15:39.3898423Z inflating: build/bin/kernel_lambda_test 2024-08-20T22:15:39.3960035Z inflating: build/bin/kernel_stackbased_test 2024-08-20T22:15:39.4057005Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T22:15:39.4108608Z inflating: build/bin/CppSignature_test 2024-08-20T22:15:39.4158143Z inflating: build/bin/op_allowlist_test 2024-08-20T22:15:39.4214275Z inflating: build/bin/backend_fallback_test 2024-08-20T22:15:39.4518395Z inflating: build/bin/op_registration_test 2024-08-20T22:15:39.4583446Z inflating: build/bin/inline_container_test 2024-08-20T22:15:39.4636679Z inflating: build/bin/cuda_apply_test 2024-08-20T22:15:39.4689114Z inflating: build/bin/cuda_allocator_test 2024-08-20T22:15:39.4748680Z inflating: build/bin/cuda_atomic_ops_test 2024-08-20T22:15:39.4803611Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-20T22:15:39.4874195Z inflating: build/bin/cuda_complex_math_test 2024-08-20T22:15:39.4934249Z inflating: build/bin/cuda_complex_test 2024-08-20T22:15:39.4984714Z inflating: build/bin/cuda_device_test 2024-08-20T22:15:39.5041636Z inflating: build/bin/cuda_cub_test 2024-08-20T22:15:39.5093068Z inflating: build/bin/cuda_dlconvertor_test 2024-08-20T22:15:39.5158470Z inflating: build/bin/cuda_distributions_test 2024-08-20T22:15:39.5216327Z inflating: build/bin/cuda_generator_test 2024-08-20T22:15:39.5265674Z inflating: build/bin/cuda_half_test 2024-08-20T22:15:39.5317204Z inflating: build/bin/cuda_integer_divider_test 2024-08-20T22:15:39.5366837Z inflating: build/bin/cuda_optional_test 2024-08-20T22:15:39.5418767Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-20T22:15:39.5472124Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-20T22:15:39.5522289Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-20T22:15:39.5583554Z inflating: build/bin/cuda_stream_test 2024-08-20T22:15:39.5633274Z inflating: build/bin/cuda_cudnn_test 2024-08-20T22:15:39.5685797Z inflating: build/bin/cuda_vectorized_test 2024-08-20T22:15:39.5700414Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T22:15:39.5766844Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T22:15:39.5824618Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-20T22:15:39.5888898Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-20T22:15:39.5951588Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-20T22:15:39.6781313Z inflating: build/bin/test_tensorexpr 2024-08-20T22:15:39.7364551Z inflating: build/bin/test_jit 2024-08-20T22:15:39.7365058Z creating: .additional_ci_files/ 2024-08-20T22:15:39.7426591Z inflating: .additional_ci_files/test-times.json 2024-08-20T22:15:39.7666229Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T22:15:39.7702490Z ##[group]Run rm artifacts.zip 2024-08-20T22:15:39.7702891Z rm artifacts.zip 2024-08-20T22:15:39.7711893Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:39.7712396Z env: 2024-08-20T22:15:39.7712676Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:39.7713139Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:39.7713634Z ##[endgroup] 2024-08-20T22:15:39.8809207Z ##[group]Run df -H 2024-08-20T22:15:39.8809759Z df -H 2024-08-20T22:15:39.8818506Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:39.8819013Z env: 2024-08-20T22:15:39.8819296Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:39.8819758Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:39.8820250Z ##[endgroup] 2024-08-20T22:15:39.8877310Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T22:15:39.8878072Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T22:15:39.8878701Z tmpfs 34G 0 34G 0% /dev/shm 2024-08-20T22:15:39.8879275Z tmpfs 14G 553k 14G 1% /run 2024-08-20T22:15:39.8879816Z /dev/nvme0n1p1 161G 85G 77G 53% / 2024-08-20T22:15:39.8880575Z tmpfs 34G 1.7M 34G 1% /tmp 2024-08-20T22:15:39.8881175Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T22:15:39.8881722Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-08-20T22:15:39.8923039Z Prepare all required actions 2024-08-20T22:15:39.8923499Z Getting action download info 2024-08-20T22:15:40.0393290Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T22:15:40.0393777Z with: 2024-08-20T22:15:40.0394035Z env: 2024-08-20T22:15:40.0394317Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:40.0394791Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:40.0395293Z ##[endgroup] 2024-08-20T22:15:40.0428908Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:15:40.0429370Z with: 2024-08-20T22:15:40.0429644Z name: td_results 2024-08-20T22:15:40.0429958Z s3-bucket: gha-artifacts 2024-08-20T22:15:40.0430316Z region: us-east-1 2024-08-20T22:15:40.0430620Z env: 2024-08-20T22:15:40.0430927Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:40.0431408Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:40.0431907Z ##[endgroup] 2024-08-20T22:15:40.5149377Z (node:375274) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:15:40.5150319Z 2024-08-20T22:15:40.5150599Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:15:40.5151347Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:15:40.5152257Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:15:40.5704639Z Found 0 objects with prefix pytorch/pytorch/10479310721/td_results/ 2024-08-20T22:15:40.5710970Z Artifact download has finished successfully 2024-08-20T22:15:40.6032524Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T22:15:40.6033020Z mkdir -p .additional_ci_files 2024-08-20T22:15:40.6033604Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T22:15:40.6043580Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:40.6044095Z env: 2024-08-20T22:15:40.6044374Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:40.6044848Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:40.6045359Z ##[endgroup] 2024-08-20T22:15:40.6133843Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T22:15:40.6147932Z ##[error]Process completed with exit code 1. 2024-08-20T22:15:40.6183772Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T22:15:40.6184273Z .github/scripts/parse_ref.py 2024-08-20T22:15:40.6193041Z shell: /usr/bin/bash -e {0} 2024-08-20T22:15:40.6193406Z env: 2024-08-20T22:15:40.6193689Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:40.6194165Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:40.6194667Z ##[endgroup] 2024-08-20T22:15:40.6503179Z Prepare all required actions 2024-08-20T22:15:40.6543856Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T22:15:40.6544332Z with: 2024-08-20T22:15:40.6544894Z github-token: *** 2024-08-20T22:15:40.6545210Z env: 2024-08-20T22:15:40.6545491Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:40.6545966Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:40.6546676Z ##[endgroup] 2024-08-20T22:15:40.6564595Z ##[group]Run set -eux 2024-08-20T22:15:40.6564941Z set -eux 2024-08-20T22:15:40.6565549Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T22:15:40.6574417Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:40.6574922Z env: 2024-08-20T22:15:40.6575202Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:40.6575659Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:40.6576315Z GITHUB_TOKEN: *** 2024-08-20T22:15:40.6576628Z ##[endgroup] 2024-08-20T22:15:40.6604545Z + python3 .github/scripts/get_workflow_job_id.py 10479310721 i-024014f1fb452f446 2024-08-20T22:15:41.2958359Z setting job-id=29026298067 2024-08-20T22:15:41.2959439Z setting job-name=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:41.3182751Z Prepare all required actions 2024-08-20T22:15:41.3183201Z Getting action download info 2024-08-20T22:15:41.4445294Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T22:15:41.4445753Z with: 2024-08-20T22:15:41.4446213Z github-token: *** 2024-08-20T22:15:41.4453266Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:15:41.4460924Z job-name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:41.4461897Z env: 2024-08-20T22:15:41.4462170Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:41.4462632Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:41.4463118Z ##[endgroup] 2024-08-20T22:15:41.4506126Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:15:41.4506686Z with: 2024-08-20T22:15:41.4506951Z shell: bash 2024-08-20T22:15:41.4507251Z timeout_minutes: 10 2024-08-20T22:15:41.4507569Z max_attempts: 5 2024-08-20T22:15:41.4507885Z retry_wait_seconds: 30 2024-08-20T22:15:41.4509015Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:15:41.4510207Z polling_interval_seconds: 1 2024-08-20T22:15:41.4510580Z warning_on_retry: true 2024-08-20T22:15:41.4510921Z continue_on_error: false 2024-08-20T22:15:41.4511263Z env: 2024-08-20T22:15:41.4511537Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:41.4512177Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:41.4512834Z GITHUB_TOKEN: *** 2024-08-20T22:15:41.4513149Z ##[endgroup] 2024-08-20T22:15:41.6899025Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:15:42.0665984Z Collecting requests==2.27.1 2024-08-20T22:15:42.0679534Z Using cached requests-2.27.1-py2.py3-none-any.whl.metadata (5.0 kB) 2024-08-20T22:15:42.1067434Z Collecting pyyaml==6.0.1 2024-08-20T22:15:42.1083670Z Using cached PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB) 2024-08-20T22:15:42.1483584Z Collecting urllib3<1.27,>=1.21.1 (from requests==2.27.1) 2024-08-20T22:15:42.1500358Z Using cached urllib3-1.26.19-py2.py3-none-any.whl.metadata (49 kB) 2024-08-20T22:15:42.1572249Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/miniconda/lib/python3.12/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-20T22:15:42.2211306Z Collecting charset-normalizer~=2.0.0 (from requests==2.27.1) 2024-08-20T22:15:42.2226572Z Using cached charset_normalizer-2.0.12-py3-none-any.whl.metadata (11 kB) 2024-08-20T22:15:42.2255405Z Requirement already satisfied: idna<4,>=2.5 in /home/ec2-user/miniconda/lib/python3.12/site-packages (from requests==2.27.1) (3.7) 2024-08-20T22:15:42.2318908Z Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-20T22:15:42.2343531Z Using cached PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (724 kB) 2024-08-20T22:15:42.2388607Z Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-20T22:15:42.2408477Z Using cached urllib3-1.26.19-py2.py3-none-any.whl (143 kB) 2024-08-20T22:15:42.3049549Z Installing collected packages: urllib3, pyyaml, charset-normalizer, requests 2024-08-20T22:15:42.3051083Z Attempting uninstall: urllib3 2024-08-20T22:15:42.3070988Z Found existing installation: urllib3 2.2.2 2024-08-20T22:15:42.3115418Z Uninstalling urllib3-2.2.2: 2024-08-20T22:15:42.3124265Z Successfully uninstalled urllib3-2.2.2 2024-08-20T22:15:42.4254684Z Attempting uninstall: charset-normalizer 2024-08-20T22:15:42.4288409Z Found existing installation: charset-normalizer 3.3.2 2024-08-20T22:15:42.4320763Z Uninstalling charset-normalizer-3.3.2: 2024-08-20T22:15:42.4338949Z Successfully uninstalled charset-normalizer-3.3.2 2024-08-20T22:15:42.4591523Z Attempting uninstall: requests 2024-08-20T22:15:42.4617320Z Found existing installation: requests 2.32.3 2024-08-20T22:15:42.4645983Z Uninstalling requests-2.32.3: 2024-08-20T22:15:42.4652500Z Successfully uninstalled requests-2.32.3 2024-08-20T22:15:42.5009796Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:15:42.5011231Z conda 24.7.1 requires requests<3,>=2.28.0, but you have requests 2.27.1 which is incompatible. 2024-08-20T22:15:42.5012394Z Successfully installed charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 urllib3-1.26.19 2024-08-20T22:15:43.5066652Z Command completed after 1 attempt(s). 2024-08-20T22:15:43.5127141Z ##[group]Run set -x 2024-08-20T22:15:43.5127476Z set -x 2024-08-20T22:15:43.5127774Z  2024-08-20T22:15:43.5128332Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:15:43.5129022Z # in runner workspace 2024-08-20T22:15:43.5129561Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T22:15:43.5139457Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:43.5139970Z env: 2024-08-20T22:15:43.5140244Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:43.5140707Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:43.5141338Z ##[endgroup] 2024-08-20T22:15:43.5171784Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T22:15:43.5413523Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:15:43.5414282Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:15:43.5414757Z echo "Job name: ${JOB_NAME}" 2024-08-20T22:15:43.5415162Z  2024-08-20T22:15:43.5415714Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:15:43.5416411Z # in runner workspace 2024-08-20T22:15:43.5416995Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T22:15:43.5417649Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T22:15:43.5418110Z  --job-name "${JOB_NAME}" \ 2024-08-20T22:15:43.5425449Z  --test-matrix "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-08-20T22:15:43.5432834Z  --selected-test-configs "" \ 2024-08-20T22:15:43.5433300Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T22:15:43.5433726Z  --tag "${TAG}" \ 2024-08-20T22:15:43.5434123Z  --event-name "${EVENT_NAME}" \ 2024-08-20T22:15:43.5434569Z  --schedule "${SCHEDULE}" \ 2024-08-20T22:15:43.5435001Z  --branch "${HEAD_BRANCH}" 2024-08-20T22:15:43.5443780Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:43.5444287Z env: 2024-08-20T22:15:43.5444561Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:43.5445027Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:43.5445708Z GITHUB_TOKEN: *** 2024-08-20T22:15:43.5446560Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:43.5447855Z PR_NUMBER: 2024-08-20T22:15:43.5448168Z TAG: ciflow/inductor/133712 2024-08-20T22:15:43.5448536Z EVENT_NAME: push 2024-08-20T22:15:43.5448841Z SCHEDULE: 2024-08-20T22:15:43.5449120Z HEAD_BRANCH: 2024-08-20T22:15:43.5449416Z ##[endgroup] 2024-08-20T22:15:43.5479198Z Workflow: inductor-periodic 2024-08-20T22:15:43.5480298Z Job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:43.7993373Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T22:15:43.9871576Z ##[group]Run echo "Filtered matrix:" 2024-08-20T22:15:43.9872030Z echo "Filtered matrix:" 2024-08-20T22:15:43.9879058Z echo "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-08-20T22:15:43.9886354Z  2024-08-20T22:15:43.9886630Z echo 2024-08-20T22:15:43.9887013Z echo "Is the current job unstable? False" 2024-08-20T22:15:43.9887469Z  2024-08-20T22:15:43.9887740Z echo 2024-08-20T22:15:43.9888102Z echo "Is keep-going label set? False" 2024-08-20T22:15:43.9888540Z  2024-08-20T22:15:43.9888817Z echo 2024-08-20T22:15:43.9889137Z echo "Renabled issues? " 2024-08-20T22:15:43.9897906Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:43.9898404Z env: 2024-08-20T22:15:43.9898683Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:43.9899142Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:43.9899634Z ##[endgroup] 2024-08-20T22:15:43.9928640Z Filtered matrix: 2024-08-20T22:15:43.9936526Z {include: [{config: dynamo_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-08-20T22:15:43.9943607Z 2024-08-20T22:15:43.9943767Z Is the current job unstable? False 2024-08-20T22:15:43.9944059Z 2024-08-20T22:15:43.9944433Z Is keep-going label set? False 2024-08-20T22:15:43.9944700Z 2024-08-20T22:15:43.9944962Z Renabled issues? 2024-08-20T22:15:43.9986161Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:43.9986874Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:43.9995069Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:43.9995564Z env: 2024-08-20T22:15:43.9995840Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:43.9996300Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:43.9996793Z JOB_TIMEOUT: 240 2024-08-20T22:15:43.9997094Z ##[endgroup] 2024-08-20T22:15:44.0083153Z ##[group]Run set -x 2024-08-20T22:15:44.0083559Z set -x 2024-08-20T22:15:44.0083867Z  2024-08-20T22:15:44.0084233Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T22:15:44.0084791Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T22:15:44.0085374Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T22:15:44.0085905Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T22:15:44.0086350Z else 2024-08-20T22:15:44.0086696Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:15:44.0087134Z fi 2024-08-20T22:15:44.0087421Z  2024-08-20T22:15:44.0087905Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T22:15:44.0088695Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T22:15:44.0089406Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T22:15:44.0090010Z # shellcheck disable=SC2086,SC2090 2024-08-20T22:15:44.0090485Z container_name=$(docker run \ 2024-08-20T22:15:44.0090916Z  ${GPU_FLAG:-} \ 2024-08-20T22:15:44.0091301Z  -e BUILD_ENVIRONMENT \ 2024-08-20T22:15:44.0091719Z  -e PR_NUMBER \ 2024-08-20T22:15:44.0092085Z  -e GITHUB_ACTIONS \ 2024-08-20T22:15:44.0092492Z  -e GITHUB_REPOSITORY \ 2024-08-20T22:15:44.0092909Z  -e GITHUB_WORKFLOW \ 2024-08-20T22:15:44.0093315Z  -e GITHUB_JOB \ 2024-08-20T22:15:44.0093682Z  -e GITHUB_RUN_ID \ 2024-08-20T22:15:44.0094079Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T22:15:44.0094494Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T22:15:44.0094901Z  -e JOB_ID \ 2024-08-20T22:15:44.0095245Z  -e JOB_NAME \ 2024-08-20T22:15:44.0095600Z  -e BASE_SHA \ 2024-08-20T22:15:44.0095953Z  -e BRANCH \ 2024-08-20T22:15:44.0096291Z  -e SHA1 \ 2024-08-20T22:15:44.0096643Z  -e AWS_DEFAULT_REGION \ 2024-08-20T22:15:44.0097063Z  -e IN_WHEEL_TEST \ 2024-08-20T22:15:44.0097450Z  -e SHARD_NUMBER \ 2024-08-20T22:15:44.0097822Z  -e TEST_CONFIG \ 2024-08-20T22:15:44.0098206Z  -e NUM_TEST_SHARDS \ 2024-08-20T22:15:44.0098614Z  -e REENABLED_ISSUES \ 2024-08-20T22:15:44.0099039Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T22:15:44.0099472Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T22:15:44.0099894Z  -e TEST_SHOWLOCALS \ 2024-08-20T22:15:44.0100297Z  -e NO_TEST_TIMEOUT \ 2024-08-20T22:15:44.0100685Z  -e NO_TD \ 2024-08-20T22:15:44.0101029Z  -e TD_DISTRIBUTED \ 2024-08-20T22:15:44.0101560Z  -e PR_LABELS \ 2024-08-20T22:15:44.0101980Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T22:15:44.0102442Z  -e SCCACHE_BUCKET \ 2024-08-20T22:15:44.0102850Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T22:15:44.0103271Z  -e XLA_CUDA \ 2024-08-20T22:15:44.0103690Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T22:15:44.0104212Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T22:15:44.0104755Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T22:15:44.0105292Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T22:15:44.0105779Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T22:15:44.0106242Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T22:15:44.0106896Z  -e DASHBOARD_TAG \ 2024-08-20T22:15:44.0107391Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T22:15:44.0107970Z  --security-opt seccomp=unconfined \ 2024-08-20T22:15:44.0108446Z  --cap-add=SYS_PTRACE \ 2024-08-20T22:15:44.0108855Z  --ipc=host \ 2024-08-20T22:15:44.0109224Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T22:15:44.0109636Z  --tty \ 2024-08-20T22:15:44.0109953Z  --detach \ 2024-08-20T22:15:44.0110445Z  --name="${container_name}" \ 2024-08-20T22:15:44.0110880Z  --user jenkins \ 2024-08-20T22:15:44.0111384Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T22:15:44.0111961Z  -w /var/lib/jenkins/workspace \ 2024-08-20T22:15:44.0112414Z  "${DOCKER_IMAGE}" 2024-08-20T22:15:44.0112775Z ) 2024-08-20T22:15:44.0113195Z # Propagate download.pytorch.org IP to container 2024-08-20T22:15:44.0114152Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T22:15:44.0115156Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T22:15:44.0116106Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T22:15:44.0125154Z shell: /usr/bin/bash -e {0} 2024-08-20T22:15:44.0125499Z env: 2024-08-20T22:15:44.0125769Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:44.0126235Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:44.0126840Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:44.0127340Z PR_NUMBER: 2024-08-20T22:15:44.0127665Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T22:15:44.0128097Z GITHUB_WORKFLOW: inductor-periodic 2024-08-20T22:15:44.0128502Z GITHUB_JOB: test 2024-08-20T22:15:44.0128813Z GITHUB_RUN_ID: 10479310721 2024-08-20T22:15:44.0129182Z GITHUB_RUN_NUMBER: 54565 2024-08-20T22:15:44.0129538Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T22:15:44.0129875Z JOB_ID: 29026298067 2024-08-20T22:15:44.0130725Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:44.0131640Z BRANCH: 2024-08-20T22:15:44.0131975Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:44.0132504Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:44.0132995Z TEST_CONFIG: dynamo_eager_torchbench 2024-08-20T22:15:44.0133410Z SHARD_NUMBER: 2 2024-08-20T22:15:44.0133720Z NUM_TEST_SHARDS: 2 2024-08-20T22:15:44.0134046Z REENABLED_ISSUES: 2024-08-20T22:15:44.0134376Z CONTINUE_THROUGH_ERROR: False 2024-08-20T22:15:44.0134760Z VERBOSE_TEST_LOGS: False 2024-08-20T22:15:44.0135120Z TEST_SHOWLOCALS: False 2024-08-20T22:15:44.0135459Z NO_TEST_TIMEOUT: False 2024-08-20T22:15:44.0135790Z NO_TD: False 2024-08-20T22:15:44.0136089Z TD_DISTRIBUTED: False 2024-08-20T22:15:44.0136511Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T22:15:44.0137027Z SCCACHE_S3_KEY_PREFIX: inductor-periodic 2024-08-20T22:15:44.0137455Z SHM_SIZE: 2g 2024-08-20T22:15:44.0138550Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:15:44.0139735Z XLA_CUDA: 2024-08-20T22:15:44.0140212Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:15:44.0140837Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T22:15:44.0141383Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T22:15:44.0141796Z DASHBOARD_TAG: 2024-08-20T22:15:44.0142336Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T22:15:44.0142737Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T22:15:44.0143111Z ##[endgroup] 2024-08-20T22:15:44.0172906Z + [[ dynamo_eager_torchbench == \m\u\l\t\i\g\p\u ]] 2024-08-20T22:15:44.0173730Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-08-20T22:15:44.0174423Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:15:44.0182082Z +++ nproc --ignore=2 2024-08-20T22:15:44.0207278Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310721 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:15:55.5032383Z + container_name=1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T22:15:55.5035180Z + grep download.pytorch.org /etc/hosts 2024-08-20T22:15:55.5036883Z + docker exec -i 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T22:15:55.6173249Z + echo DOCKER_CONTAINER_ID=1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T22:15:55.6177730Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:15:55.6180819Z + docker exec -t 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T22:15:56.0213634Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T22:15:56.3436156Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T22:15:56.3438681Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T22:15:56.3440889Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T22:15:56.3444238Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T22:15:56.3446698Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T22:15:56.3452387Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.13.1) 2024-08-20T22:15:56.3468679Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T22:15:56.3481768Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T22:15:56.3499043Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.22.4) 2024-08-20T22:15:56.3885373Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T22:15:56.9661863Z Installing collected packages: torch 2024-08-20T22:16:06.7938930Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:16:06.7940783Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T22:16:06.7941682Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T22:16:06.8891760Z + export TERM=vt100 2024-08-20T22:16:06.8892821Z + TERM=vt100 2024-08-20T22:16:06.8895642Z ++ dirname .ci/pytorch/test.sh 2024-08-20T22:16:06.9058858Z + source .ci/pytorch/common.sh 2024-08-20T22:16:06.9063293Z +++ dirname .ci/pytorch/common.sh 2024-08-20T22:16:06.9074613Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T22:16:06.9076061Z +++ declare -f -t trap_add 2024-08-20T22:16:06.9082478Z ++ set -ex 2024-08-20T22:16:06.9082965Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:16:06.9083484Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T22:16:06.9083996Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-08-20T22:16:06.9087165Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T22:16:06.9379409Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T22:16:06.9379851Z + trap_add cleanup_workspace EXIT 2024-08-20T22:16:06.9380288Z + trap_add_cmd=cleanup_workspace 2024-08-20T22:16:06.9380667Z + shift 2024-08-20T22:16:06.9384540Z + for trap_add_name in "$@" 2024-08-20T22:16:06.9387839Z +++ trap -p EXIT 2024-08-20T22:16:06.9390310Z ++ eval 'extract_trap_cmd ' 2024-08-20T22:16:06.9390795Z +++ extract_trap_cmd 2024-08-20T22:16:06.9391207Z +++ printf '%s\n' '' 2024-08-20T22:16:06.9391618Z ++ printf '%s\n' cleanup_workspace 2024-08-20T22:16:06.9393985Z + trap -- ' 2024-08-20T22:16:06.9394436Z cleanup_workspace' EXIT 2024-08-20T22:16:06.9394980Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T22:16:07.6956407Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T22:16:07.6979350Z + echo 'Environment variables:' 2024-08-20T22:16:07.6979899Z Environment variables: 2024-08-20T22:16:07.6980318Z + env 2024-08-20T22:16:07.7179577Z INSTALLED_DB=yes 2024-08-20T22:16:07.7180339Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:16:07.7180902Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:16:07.7181381Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:16:07.7182191Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:16:07.7183187Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:16:07.7183833Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:16:07.7184477Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:16:07.7185113Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:16:07.7185665Z HOSTNAME=1409a3b58649 2024-08-20T22:16:07.7186648Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7187627Z GITHUB_ACTION=__self 2024-08-20T22:16:07.7188105Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:16:07.7192420Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:16:07.7196737Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:16:07.7197264Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:16:07.7197637Z GITHUB_RUN_NUMBER=54565 2024-08-20T22:16:07.7198007Z TEST_CONFIG=dynamo_eager_torchbench 2024-08-20T22:16:07.7198432Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:16:07.7198924Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:16:07.7199400Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:16:07.7199854Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:16:07.7200422Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:16:07.7200835Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:16:07.7201296Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:16:07.7201802Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:16:07.7202282Z GITHUB_REF_TYPE=tag 2024-08-20T22:16:07.7202613Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:16:07.7203001Z NCCL_VERSION=2.17.1-1 2024-08-20T22:16:07.7203399Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:07.7203864Z XLA_CUDA= 2024-08-20T22:16:07.7204378Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:16:07.7206597Z *** 2024-08-20T22:16:07.7206888Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:16:07.7207297Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:16:07.7207671Z GITHUB_ACTIONS=true 2024-08-20T22:16:07.7208010Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:16:07.7208561Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:16:07.7209132Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:16:07.7209656Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:07.7210182Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:16:07.7210671Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:07.7211620Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:16:07.7212402Z UCC_HOME=/usr 2024-08-20T22:16:07.7212746Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:16:07.7213158Z VERBOSE_TEST_LOGS=False 2024-08-20T22:16:07.7213503Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:16:07.7213974Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:16:07.7214482Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:16:07.7214958Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:16:07.7215335Z SHARD_NUMBER=2 2024-08-20T22:16:07.7215647Z GITHUB_REF_PROTECTED=false 2024-08-20T22:16:07.7216001Z HOME=/var/lib/jenkins 2024-08-20T22:16:07.7216364Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:16:07.7216816Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:16:07.7217297Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:16:07.7217854Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-20T22:16:07.7218266Z CUDA_VERSION=12.1.1 2024-08-20T22:16:07.7218696Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:16:07.7219140Z NUM_TEST_SHARDS=2 2024-08-20T22:16:07.7219443Z UCX_HOME=/usr 2024-08-20T22:16:07.7219965Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:16:07.7221078Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7222738Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:16:07.7224171Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7225311Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:16:07.7225983Z GITHUB_EVENT_NAME=push 2024-08-20T22:16:07.7226318Z DASHBOARD_TAG= 2024-08-20T22:16:07.7226628Z GITHUB_RUN_ID=10479310721 2024-08-20T22:16:07.7227090Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:16:07.7227637Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:16:07.7228661Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7229713Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:16:07.7230123Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:16:07.7230503Z PR_NUMBER= 2024-08-20T22:16:07.7230789Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:16:07.7231123Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:16:07.7231573Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:16:07.7232037Z TERM=vt100 2024-08-20T22:16:07.7232379Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:16:07.7232796Z INSTALLED_VISION=yes 2024-08-20T22:16:07.7233209Z BRANCH= 2024-08-20T22:16:07.7233500Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:16:07.7233910Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:16:07.7234334Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:16:07.7235109Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:16:07.7235874Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:16:07.7236367Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:16:07.7236839Z REENABLED_ISSUES= 2024-08-20T22:16:07.7237139Z SHLVL=1 2024-08-20T22:16:07.7237405Z MAX_JOBS=14 2024-08-20T22:16:07.7237716Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:16:07.7238072Z NVARCH=x86_64 2024-08-20T22:16:07.7238376Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:16:07.7238841Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:07.7239377Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:16:07.7239874Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:16:07.7240553Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:16:07.7241129Z GITHUB_JOB=test 2024-08-20T22:16:07.7241529Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:16:07.7242098Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:16:07.7242618Z NO_TEST_TIMEOUT=False 2024-08-20T22:16:07.7242943Z TD_DISTRIBUTED=False 2024-08-20T22:16:07.7243329Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:16:07.7243778Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:16:07.7244207Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:16:07.7244569Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:16:07.7244919Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:16:07.7245272Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:16:07.7246331Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:07.7249778Z GITHUB_BASE_REF= 2024-08-20T22:16:07.7250121Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:16:07.7250498Z CI=true 2024-08-20T22:16:07.7250834Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:16:07.7251252Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:16:07.7251632Z JOB_ID=29026298067 2024-08-20T22:16:07.7251952Z INSTALLED_PROTOBUF=yes 2024-08-20T22:16:07.7252285Z GITHUB_HEAD_REF= 2024-08-20T22:16:07.7252593Z GITHUB_ACTION_REF= 2024-08-20T22:16:07.7253040Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:16:07.7253527Z TEST_SHOWLOCALS=False 2024-08-20T22:16:07.7253922Z GITHUB_WORKFLOW=inductor-periodic 2024-08-20T22:16:07.7254335Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:16:07.7255276Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7256116Z NO_TD=False 2024-08-20T22:16:07.7256417Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:16:07.7256788Z _=/usr/bin/env 2024-08-20T22:16:07.7257282Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T22:16:07.7413996Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-20T22:16:07.7414944Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-20T22:16:07.7415827Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-20T22:16:07.7416749Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-20T22:16:07.7417378Z + BUILD_DIR=build 2024-08-20T22:16:07.7417978Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T22:16:07.7418401Z + BUILD_BIN_DIR=build/bin 2024-08-20T22:16:07.7418784Z + SHARD_NUMBER=2 2024-08-20T22:16:07.7419096Z + NUM_TEST_SHARDS=2 2024-08-20T22:16:07.7419426Z + export VALGRIND=ON 2024-08-20T22:16:07.7419743Z + VALGRIND=ON 2024-08-20T22:16:07.7420233Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-08-20T22:16:07.7420733Z + [[ 0 == \1 ]] 2024-08-20T22:16:07.7421027Z + [[ False == \1 ]] 2024-08-20T22:16:07.7422744Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-08-20T22:16:07.7423303Z ++ realpath build/custom_test_artifacts 2024-08-20T22:16:07.7745648Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T22:16:07.7746378Z + [[ -n '' ]] 2024-08-20T22:16:07.7746734Z + echo 'Environment variables' 2024-08-20T22:16:07.7747441Z Environment variables 2024-08-20T22:16:07.7747766Z + env 2024-08-20T22:16:07.7756644Z INSTALLED_DB=yes 2024-08-20T22:16:07.7757043Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:16:07.7757455Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:16:07.7757903Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:16:07.7758982Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:16:07.7759749Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:16:07.7760418Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:16:07.7760968Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:16:07.7761677Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:16:07.7762345Z HOSTNAME=1409a3b58649 2024-08-20T22:16:07.7763334Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7764160Z GITHUB_ACTION=__self 2024-08-20T22:16:07.7764514Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:16:07.7768639Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:16:07.7772830Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:16:07.7773346Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:16:07.7773698Z GITHUB_RUN_NUMBER=54565 2024-08-20T22:16:07.7774061Z TEST_CONFIG=dynamo_eager_torchbench 2024-08-20T22:16:07.7774483Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:16:07.7774969Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:16:07.7775435Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:16:07.7775885Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:16:07.7776299Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:16:07.7776804Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:16:07.7777257Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:16:07.7777760Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:16:07.7778231Z GITHUB_REF_TYPE=tag 2024-08-20T22:16:07.7778554Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:16:07.7779031Z NCCL_VERSION=2.17.1-1 2024-08-20T22:16:07.7779426Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:07.7779879Z XLA_CUDA= 2024-08-20T22:16:07.7780389Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:16:07.7780793Z *** 2024-08-20T22:16:07.7781074Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:16:07.7781474Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:16:07.7781952Z GITHUB_ACTIONS=true 2024-08-20T22:16:07.7782287Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:16:07.7782987Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:16:07.7783533Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:16:07.7784004Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:07.7784526Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:16:07.7785018Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:07.7785978Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:16:07.7786754Z UCC_HOME=/usr 2024-08-20T22:16:07.7787229Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:16:07.7787637Z VERBOSE_TEST_LOGS=False 2024-08-20T22:16:07.7787977Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:16:07.7788476Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:16:07.7789011Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:16:07.7789487Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:16:07.7789864Z SHARD_NUMBER=2 2024-08-20T22:16:07.7790179Z GITHUB_REF_PROTECTED=false 2024-08-20T22:16:07.7790542Z HOME=/var/lib/jenkins 2024-08-20T22:16:07.7790912Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:16:07.7791404Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:16:07.7791881Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:16:07.7792446Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-20T22:16:07.7792868Z CUDA_VERSION=12.1.1 2024-08-20T22:16:07.7793289Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:16:07.7793735Z NUM_TEST_SHARDS=2 2024-08-20T22:16:07.7794038Z UCX_HOME=/usr 2024-08-20T22:16:07.7794574Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:16:07.7795680Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7797157Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:16:07.7798596Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7799749Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:16:07.7800415Z GITHUB_EVENT_NAME=push 2024-08-20T22:16:07.7800746Z DASHBOARD_TAG= 2024-08-20T22:16:07.7801050Z GITHUB_RUN_ID=10479310721 2024-08-20T22:16:07.7801519Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:16:07.7802060Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:16:07.7803091Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7804023Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:16:07.7804435Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:16:07.7804812Z PR_NUMBER= 2024-08-20T22:16:07.7805097Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:16:07.7805415Z VALGRIND=ON 2024-08-20T22:16:07.7805713Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:16:07.7806153Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:16:07.7806619Z TERM=vt100 2024-08-20T22:16:07.7806957Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:16:07.7807376Z INSTALLED_VISION=yes 2024-08-20T22:16:07.7807685Z BRANCH= 2024-08-20T22:16:07.7807972Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:16:07.7808376Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:16:07.7808796Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:16:07.7809573Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:16:07.7810349Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:16:07.7810845Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:16:07.7811316Z REENABLED_ISSUES= 2024-08-20T22:16:07.7811619Z SHLVL=1 2024-08-20T22:16:07.7811883Z MAX_JOBS=14 2024-08-20T22:16:07.7812200Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:16:07.7812550Z NVARCH=x86_64 2024-08-20T22:16:07.7812849Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:16:07.7813311Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:07.7813947Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:16:07.7814451Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:16:07.7815129Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:16:07.7815715Z GITHUB_JOB=test 2024-08-20T22:16:07.7816113Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:16:07.7816679Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:16:07.7817200Z NO_TEST_TIMEOUT=False 2024-08-20T22:16:07.7817661Z TD_DISTRIBUTED=False 2024-08-20T22:16:07.7818047Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:16:07.7818494Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:16:07.7818921Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:16:07.7819282Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:16:07.7819635Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:16:07.7819990Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:16:07.7821058Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:07.7822228Z GITHUB_BASE_REF= 2024-08-20T22:16:07.7822559Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:16:07.7822935Z CI=true 2024-08-20T22:16:07.7823259Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:16:07.7823671Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:16:07.7824053Z JOB_ID=29026298067 2024-08-20T22:16:07.7824377Z INSTALLED_PROTOBUF=yes 2024-08-20T22:16:07.7824714Z GITHUB_HEAD_REF= 2024-08-20T22:16:07.7825021Z GITHUB_ACTION_REF= 2024-08-20T22:16:07.7825466Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:16:07.7825938Z TEST_SHOWLOCALS=False 2024-08-20T22:16:07.7826328Z GITHUB_WORKFLOW=inductor-periodic 2024-08-20T22:16:07.7826739Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:16:07.7827668Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_4ecd6160-6d2e-4f0f-92f7-edd965f406ce 2024-08-20T22:16:07.7828537Z NO_TD=False 2024-08-20T22:16:07.7828861Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:16:07.7829227Z _=/usr/bin/env 2024-08-20T22:16:07.7829571Z + echo 'Testing pytorch' 2024-08-20T22:16:07.7829918Z Testing pytorch 2024-08-20T22:16:07.7830238Z + export LANG=C.UTF-8 2024-08-20T22:16:07.7830581Z + LANG=C.UTF-8 2024-08-20T22:16:07.8791492Z + PR_NUMBER= 2024-08-20T22:16:07.8791918Z + [[ dynamo_eager_torchbench == \d\e\f\a\u\l\t ]] 2024-08-20T22:16:07.8792671Z + [[ dynamo_eager_torchbench == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T22:16:07.8793371Z + [[ dynamo_eager_torchbench == \s\l\o\w ]] 2024-08-20T22:16:07.8794308Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-08-20T22:16:07.8795265Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-08-20T22:16:07.8795861Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:16:07.8796344Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:16:07.8796810Z + [[ dynamo_eager_torchbench == *crossref* ]] 2024-08-20T22:16:07.8797402Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:16:07.8798068Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-08-20T22:16:07.8798723Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:16:07.8799293Z + pip_install --user ninja==1.10.2 2024-08-20T22:16:07.8799833Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T22:16:08.7364081Z Collecting ninja==1.10.2 2024-08-20T22:16:08.7534418Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T22:16:08.8051524Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T22:16:09.4010092Z Installing collected packages: ninja 2024-08-20T22:16:09.4089782Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T22:16:09.4092583Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T22:16:09.4730995Z Successfully installed ninja-1.10.2 2024-08-20T22:16:09.5505169Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:09.5507671Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:09.5509359Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:16:09.5509877Z + install_tlparse 2024-08-20T22:16:09.5510281Z + pip_install --user tlparse==0.3.25 2024-08-20T22:16:09.5510863Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T22:16:09.9755831Z Collecting tlparse==0.3.25 2024-08-20T22:16:09.9906201Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T22:16:10.0468798Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T22:16:10.6872571Z Installing collected packages: tlparse 2024-08-20T22:16:10.7639904Z Successfully installed tlparse-0.3.25 2024-08-20T22:16:10.8409999Z ++ python -m site --user-base 2024-08-20T22:16:10.8654524Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:10.8657826Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-08-20T22:16:10.8659174Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-08-20T22:16:10.8659873Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:16:10.8660834Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-08-20T22:16:10.8662096Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-08-20T22:16:10.8662823Z + cd test 2024-08-20T22:16:10.8663365Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T22:16:13.0541015Z + [[ dynamo_eager_torchbench == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T22:16:13.0541851Z + [[ dynamo_eager_torchbench == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T22:16:13.0544385Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T22:16:13.0545584Z + [[ dynamo_eager_torchbench == *pr_time_benchmarks* ]] 2024-08-20T22:16:13.0546296Z + [[ dynamo_eager_torchbench == *dynamo_eager* ]] 2024-08-20T22:16:13.0547515Z + DYNAMO_BENCHMARK_FLAGS+=(--backend eager) 2024-08-20T22:16:13.0548014Z + [[ dynamo_eager_torchbench == *dynamic* ]] 2024-08-20T22:16:13.0548500Z + [[ dynamo_eager_torchbench == *cpu* ]] 2024-08-20T22:16:13.0549020Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T22:16:13.0580863Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:16:13.0581980Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-08-20T22:16:13.0585346Z + cd test 2024-08-20T22:16:13.0586034Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T22:16:14.5165183Z PyTorch built with: 2024-08-20T22:16:14.5166491Z - GCC 9.4 2024-08-20T22:16:14.5167319Z - C++ Version: 201703 2024-08-20T22:16:14.5169125Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:16:14.5170816Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:16:14.5171529Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:16:14.5172060Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T22:16:14.5172550Z - NNPACK is enabled 2024-08-20T22:16:14.5172945Z - CPU capability usage: AVX2 2024-08-20T22:16:14.5173367Z - CUDA Runtime 12.1 2024-08-20T22:16:14.5173894Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-08-20T22:16:14.5174520Z - CuDNN 90.1 (built against CUDA 12.4) 2024-08-20T22:16:14.5175352Z - Magma 2.6.1 2024-08-20T22:16:14.5182672Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T22:16:14.5189151Z 2024-08-20T22:16:14.8333902Z + cd test 2024-08-20T22:16:14.8334686Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T22:16:16.1791667Z ATen/Parallel: 2024-08-20T22:16:16.1792227Z at::get_num_threads() : 8 2024-08-20T22:16:16.1792777Z at::get_num_interop_threads() : 16 2024-08-20T22:16:16.1793238Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:16:16.1793648Z omp_get_max_threads() : 8 2024-08-20T22:16:16.1794738Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:16:16.1795560Z mkl_get_max_threads() : 8 2024-08-20T22:16:16.1796205Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:16:16.1796871Z std::thread::hardware_concurrency() : 16 2024-08-20T22:16:16.1797308Z Environment variables: 2024-08-20T22:16:16.1797658Z OMP_NUM_THREADS : [not set] 2024-08-20T22:16:16.1798047Z MKL_NUM_THREADS : [not set] 2024-08-20T22:16:16.1798434Z ATen parallel backend: OpenMP 2024-08-20T22:16:16.1798693Z 2024-08-20T22:16:16.4573522Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:16:16.4574253Z + [[ dynamo_eager_torchbench == *backward* ]] 2024-08-20T22:16:16.4574765Z + [[ dynamo_eager_torchbench == *xla* ]] 2024-08-20T22:16:16.4575242Z + [[ dynamo_eager_torchbench == *executorch* ]] 2024-08-20T22:16:16.4575772Z + [[ dynamo_eager_torchbench == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T22:16:16.4576444Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:16:16.4577013Z + [[ dynamo_eager_torchbench == distributed ]] 2024-08-20T22:16:16.4577549Z + [[ dynamo_eager_torchbench == *inductor_distributed* ]] 2024-08-20T22:16:16.4578170Z + [[ dynamo_eager_torchbench == *inductor-halide* ]] 2024-08-20T22:16:16.4578834Z + [[ dynamo_eager_torchbench == *inductor-micro-benchmark* ]] 2024-08-20T22:16:16.4579413Z + [[ dynamo_eager_torchbench == *huggingface* ]] 2024-08-20T22:16:16.4579894Z + [[ dynamo_eager_torchbench == *timm* ]] 2024-08-20T22:16:16.4580375Z + [[ dynamo_eager_torchbench == *torchbench* ]] 2024-08-20T22:16:16.4580856Z + [[ dynamo_eager_torchbench == *cpu* ]] 2024-08-20T22:16:16.4581285Z + install_torchaudio cuda 2024-08-20T22:16:16.4581725Z + local commit 2024-08-20T22:16:16.4582051Z ++ get_pinned_commit audio 2024-08-20T22:16:16.4582432Z ++ cat .github/ci_commit_pins/audio.txt 2024-08-20T22:16:16.4603786Z + commit=b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:16.4604266Z + [[ cuda == \c\u\d\a ]] 2024-08-20T22:16:16.4604686Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-20T22:16:16.4605628Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:16.4607537Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:16.7956756Z Collecting git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:16.7962500Z Cloning https://github.com/pytorch/audio.git (to revision b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862) to /tmp/pip-req-build-ifhvashs 2024-08-20T22:16:16.7993930Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio.git /tmp/pip-req-build-ifhvashs 2024-08-20T22:16:17.6133203Z Running command git rev-parse -q --verify 'sha^b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862' 2024-08-20T22:16:17.6163324Z Running command git fetch -q https://github.com/pytorch/audio.git b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:18.0166337Z Resolved https://github.com/pytorch/audio.git to commit b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:18.0168598Z Running command git submodule update --init --recursive -q 2024-08-20T22:16:20.4636886Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:16:20.4667390Z [?25hRequirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchaudio==2.4.0a0+b3f6f51) (2.5.0a0+git40ec5f6) 2024-08-20T22:16:20.4695067Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.13.1) 2024-08-20T22:16:20.4699120Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (4.12.2) 2024-08-20T22:16:20.4702452Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2.8.8) 2024-08-20T22:16:20.4706191Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.1.4) 2024-08-20T22:16:20.4709282Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2024.6.1) 2024-08-20T22:16:20.4714662Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (1.13.1) 2024-08-20T22:16:20.4729358Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchaudio==2.4.0a0+b3f6f51) (1.3.0) 2024-08-20T22:16:20.5201316Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchaudio==2.4.0a0+b3f6f51) (2.1.5) 2024-08-20T22:16:20.5265397Z Building wheels for collected packages: torchaudio 2024-08-20T22:18:55.0809560Z Building wheel for torchaudio (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done 2024-08-20T22:18:55.0845167Z [?25h Created wheel for torchaudio: filename=torchaudio-2.4.0a0+b3f6f51-cp310-cp310-linux_x86_64.whl size=2310386 sha256=55b66b11ed18e5686546d64fe764d99951a35ec9e6b17c58ee439ebad40fc2e7 2024-08-20T22:18:55.0846876Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/b5/02/94/08dddd0295a3ebec4d4a4d9f226adfd865d48fb8ae4af2cc95 2024-08-20T22:18:55.0877231Z Successfully built torchaudio 2024-08-20T22:18:55.5774737Z Installing collected packages: torchaudio 2024-08-20T22:18:55.7907740Z Successfully installed torchaudio-2.4.0a0+b3f6f51 2024-08-20T22:18:55.9708384Z + install_torchtext 2024-08-20T22:18:55.9708877Z + local data_commit 2024-08-20T22:18:55.9709295Z + local text_commit 2024-08-20T22:18:55.9716150Z ++ get_pinned_commit data 2024-08-20T22:18:55.9716672Z ++ cat .github/ci_commit_pins/data.txt 2024-08-20T22:18:55.9734249Z + data_commit=11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:55.9737876Z ++ get_pinned_commit text 2024-08-20T22:18:55.9738362Z ++ cat .github/ci_commit_pins/text.txt 2024-08-20T22:18:55.9758110Z + text_commit=b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:18:55.9759570Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:55.9761351Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:56.3404416Z Collecting git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:56.3408383Z Cloning https://github.com/pytorch/data.git (to revision 11bb5b847ea8b9e0d9bb82db3304daf383008d3f) to /tmp/pip-req-build-g9vyrj70 2024-08-20T22:18:56.3443440Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/data.git /tmp/pip-req-build-g9vyrj70 2024-08-20T22:18:56.8087500Z Running command git rev-parse -q --verify 'sha^11bb5b847ea8b9e0d9bb82db3304daf383008d3f' 2024-08-20T22:18:56.8114556Z Running command git fetch -q https://github.com/pytorch/data.git 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:57.0662071Z Running command git checkout -q 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:57.2193417Z Resolved https://github.com/pytorch/data.git to commit 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:57.2194544Z Running command git submodule update --init --recursive -q 2024-08-20T22:20:21.5367706Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:20:21.5405054Z [?25hRequirement already satisfied: urllib3>=1.25 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (1.26.19) 2024-08-20T22:20:21.5408430Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.32.3) 2024-08-20T22:20:21.5412889Z Requirement already satisfied: torch>2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.5.0a0+git40ec5f6) 2024-08-20T22:20:21.5443327Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.13.1) 2024-08-20T22:20:21.5447997Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (4.12.2) 2024-08-20T22:20:21.5451812Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.8.8) 2024-08-20T22:20:21.5455087Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.1.4) 2024-08-20T22:20:21.5458461Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2024.6.1) 2024-08-20T22:20:21.5464013Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.13.1) 2024-08-20T22:20:21.5478651Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.3.0) 2024-08-20T22:20:21.5538214Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.3.2) 2024-08-20T22:20:21.5542463Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.7) 2024-08-20T22:20:21.5547817Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (2024.7.4) 2024-08-20T22:20:21.6048084Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.1.5) 2024-08-20T22:20:21.6112782Z Building wheels for collected packages: torchdata 2024-08-20T22:20:23.5894389Z Building wheel for torchdata (setup.py) ... [?25l- \ | done 2024-08-20T22:20:23.5902706Z [?25h Created wheel for torchdata: filename=torchdata-0.7.0a0+11bb5b8-py3-none-any.whl size=182970 sha256=d508b06be36bb87dadd04443125ffc215372a7d55e64f16260036da9b936098e 2024-08-20T22:20:23.5904308Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/ca/59/a3/c8250bfc8d3d4d639498d4beb2e0f0e70b9a508ac61fde85ce 2024-08-20T22:20:23.5933011Z Successfully built torchdata 2024-08-20T22:20:24.0766286Z Installing collected packages: torchdata 2024-08-20T22:20:24.1716615Z Successfully installed torchdata-0.7.0a0+11bb5b8 2024-08-20T22:20:29.1468721Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:29.1470265Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:29.4856778Z Collecting git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:29.4861341Z Cloning https://github.com/pytorch/text.git (to revision b0ebddc648d279826089db91775375221777a2db) to /tmp/pip-req-build-gmbplrap 2024-08-20T22:20:29.4890043Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/text.git /tmp/pip-req-build-gmbplrap 2024-08-20T22:20:30.3148411Z Running command git rev-parse -q --verify 'sha^b0ebddc648d279826089db91775375221777a2db' 2024-08-20T22:20:30.3177177Z Running command git fetch -q https://github.com/pytorch/text.git b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:30.8053485Z Running command git checkout -q b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:30.9753934Z Resolved https://github.com/pytorch/text.git to commit b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:30.9755074Z Running command git submodule update --init --recursive -q 2024-08-20T22:20:38.4614138Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:20:38.4649874Z [?25hRequirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (4.66.5) 2024-08-20T22:20:38.4653127Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.32.3) 2024-08-20T22:20:38.4655236Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.5.0a0+git40ec5f6) 2024-08-20T22:20:38.4657644Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (1.22.4) 2024-08-20T22:20:38.4661024Z Requirement already satisfied: torchdata in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (0.7.0a0+11bb5b8) 2024-08-20T22:20:38.4686872Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.3.2) 2024-08-20T22:20:38.4690456Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.7) 2024-08-20T22:20:38.4694337Z Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (1.26.19) 2024-08-20T22:20:38.4699540Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (2024.7.4) 2024-08-20T22:20:38.4725809Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.13.1) 2024-08-20T22:20:38.4728912Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (4.12.2) 2024-08-20T22:20:38.4732052Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2.8.8) 2024-08-20T22:20:38.4735656Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.1.4) 2024-08-20T22:20:38.4738160Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2024.6.1) 2024-08-20T22:20:38.4743486Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (1.13.1) 2024-08-20T22:20:38.4758324Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchtext==0.16.0a0+b0ebddc) (1.3.0) 2024-08-20T22:20:38.5337839Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchtext==0.16.0a0+b0ebddc) (2.1.5) 2024-08-20T22:20:38.5404052Z Building wheels for collected packages: torchtext 2024-08-20T22:21:16.8463533Z Building wheel for torchtext (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done 2024-08-20T22:21:16.8495123Z [?25h Created wheel for torchtext: filename=torchtext-0.16.0a0+b0ebddc-cp310-cp310-linux_x86_64.whl size=2045465 sha256=4b2bb7df65c609fcad0106984e5e33d6e577930196dfa77866932cf167dbdee9 2024-08-20T22:21:16.8496776Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/e5/3d/d7/faafad098ec9437ecdf4495c0f5e72b817fffbb06063e32cbf 2024-08-20T22:21:16.8522344Z Successfully built torchtext 2024-08-20T22:21:17.3375178Z Installing collected packages: torchtext 2024-08-20T22:21:17.4487229Z Successfully installed torchtext-0.16.0a0+b0ebddc 2024-08-20T22:21:17.6104220Z + install_torchvision 2024-08-20T22:21:17.6104634Z + local orig_preload 2024-08-20T22:21:17.6104968Z + local commit 2024-08-20T22:21:17.6107979Z ++ get_pinned_commit vision 2024-08-20T22:21:17.6108447Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T22:21:17.6127137Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:17.6127599Z + orig_preload= 2024-08-20T22:21:17.6128006Z + '[' -n '' ']' 2024-08-20T22:21:17.6128868Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:17.6130359Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:17.9516935Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:17.9521799Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-x_1c4a9z 2024-08-20T22:21:17.9549726Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-x_1c4a9z 2024-08-20T22:21:19.3977407Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T22:21:19.4005946Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:20.7342941Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:21.0368215Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:23.5250913Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-20T22:21:23.5282989Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2024-08-20T22:21:23.5287241Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T22:21:23.5293026Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T22:21:23.5362696Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T22:21:23.5367199Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T22:21:23.5370471Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T22:21:23.5374186Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T22:21:23.5377342Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T22:21:23.5382668Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-20T22:21:23.5397552Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T22:21:23.5875064Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T22:21:23.5938141Z Building wheels for collected packages: torchvision 2024-08-20T22:22:40.5927899Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-08-20T22:22:40.5961122Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2116150 sha256=d5e31f463b809d8c71f0dabc07c2f3619fb175c6c99aa8dac4566f0364f7dd03 2024-08-20T22:22:40.5962846Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-20T22:22:40.5998253Z Successfully built torchvision 2024-08-20T22:22:41.0890197Z Installing collected packages: torchvision 2024-08-20T22:22:41.5028482Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T22:22:41.6460434Z + '[' -n '' ']' 2024-08-20T22:22:41.6460889Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-20T22:22:41.6461409Z + pip_install git+https://github.com/pytorch/ao.git 2024-08-20T22:22:41.6462228Z + pip install --progress-bar off git+https://github.com/pytorch/ao.git 2024-08-20T22:22:41.9874906Z Collecting git+https://github.com/pytorch/ao.git 2024-08-20T22:22:41.9879626Z Cloning https://github.com/pytorch/ao.git to /tmp/pip-req-build-yvllmcmn 2024-08-20T22:22:41.9910089Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/ao.git /tmp/pip-req-build-yvllmcmn 2024-08-20T22:22:42.5660932Z Resolved https://github.com/pytorch/ao.git to commit 2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T22:22:44.9873291Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:22:44.9909209Z [?25hBuilding wheels for collected packages: torchao 2024-08-20T22:24:03.1670469Z Building wheel for torchao (setup.py) ... [?25l- \ | / - \ | / done 2024-08-20T22:24:03.1688648Z [?25h Created wheel for torchao: filename=torchao-0.4.0+git2c8e3f3-cp310-cp310-linux_x86_64.whl size=928019 sha256=62bd2aa25612709b8e050ceb8572e7221c685f81df22f6b32c6426b03b3b0eff 2024-08-20T22:24:03.1691005Z Stored in directory: /tmp/pip-ephem-wheel-cache-9godatat/wheels/c0/df/d8/b1e2f5f1ea3bf141268a83a667137647b234f04c636e1e7a5e 2024-08-20T22:24:03.1721535Z Successfully built torchao 2024-08-20T22:24:03.7176327Z Installing collected packages: torchao 2024-08-20T22:24:03.9569756Z Successfully installed torchao-0.4.0+git2c8e3f3 2024-08-20T22:24:04.0676669Z + id=1 2024-08-20T22:24:04.0677822Z + pip_install opencv-python==4.8.0.74 2024-08-20T22:24:04.0678441Z + pip install --progress-bar off opencv-python==4.8.0.74 2024-08-20T22:24:04.5392778Z Collecting opencv-python==4.8.0.74 2024-08-20T22:24:04.5571031Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB) 2024-08-20T22:24:04.5645090Z Requirement already satisfied: numpy>=1.21.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opencv-python==4.8.0.74) (1.22.4) 2024-08-20T22:24:04.5687649Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB) 2024-08-20T22:24:05.6219625Z Installing collected packages: opencv-python 2024-08-20T22:24:06.4856369Z Successfully installed opencv-python-4.8.0.74 2024-08-20T22:24:06.5848402Z + [[ dynamo_eager_torchbench == *inductor_torchbench_smoketest_perf* ]] 2024-08-20T22:24:06.5849190Z + [[ dynamo_eager_torchbench == *inductor_torchbench_cpu_smoketest_perf* ]] 2024-08-20T22:24:06.5850845Z + [[ dynamo_eager_torchbench == *torchbench_gcp_smoketest* ]] 2024-08-20T22:24:06.5851515Z + checkout_install_torchbench 2024-08-20T22:24:06.5851943Z + local commit 2024-08-20T22:24:06.5852586Z ++ get_pinned_commit torchbench 2024-08-20T22:24:06.5853040Z ++ cat .github/ci_commit_pins/torchbench.txt 2024-08-20T22:24:06.5870999Z + commit=23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T22:24:06.5871670Z + git clone https://github.com/pytorch/benchmark torchbench 2024-08-20T22:24:06.5888898Z Cloning into 'torchbench'... 2024-08-20T22:24:06.7434017Z remote: Enumerating objects: 31001, done. 2024-08-20T22:24:06.7438167Z remote: Counting objects: 0% (1/1741) 2024-08-20T22:24:06.7440234Z remote: Counting objects: 1% (18/1741) 2024-08-20T22:24:06.7442943Z remote: Counting objects: 2% (35/1741) 2024-08-20T22:24:06.7443629Z remote: Counting objects: 3% (53/1741) 2024-08-20T22:24:06.7445446Z remote: Counting objects: 4% (70/1741) 2024-08-20T22:24:06.7446170Z remote: Counting objects: 5% (88/1741) 2024-08-20T22:24:06.7446809Z remote: Counting objects: 6% (105/1741) 2024-08-20T22:24:06.7447968Z remote: Counting objects: 7% (122/1741) 2024-08-20T22:24:06.7448985Z remote: Counting objects: 8% (140/1741) 2024-08-20T22:24:06.7449705Z remote: Counting objects: 9% (157/1741) 2024-08-20T22:24:06.7450714Z remote: Counting objects: 10% (175/1741) 2024-08-20T22:24:06.7451369Z remote: Counting objects: 11% (192/1741) 2024-08-20T22:24:06.7452017Z remote: Counting objects: 12% (209/1741) 2024-08-20T22:24:06.7453052Z remote: Counting objects: 13% (227/1741) 2024-08-20T22:24:06.7453772Z remote: Counting objects: 14% (244/1741) 2024-08-20T22:24:06.7454312Z remote: Counting objects: 15% (262/1741) 2024-08-20T22:24:06.7454847Z remote: Counting objects: 16% (279/1741) 2024-08-20T22:24:06.7455766Z remote: Counting objects: 17% (296/1741) 2024-08-20T22:24:06.7456399Z remote: Counting objects: 18% (314/1741) 2024-08-20T22:24:06.7457034Z remote: Counting objects: 19% (331/1741) 2024-08-20T22:24:06.7457730Z remote: Counting objects: 20% (349/1741) 2024-08-20T22:24:06.7458378Z remote: Counting objects: 21% (366/1741) 2024-08-20T22:24:06.7459089Z remote: Counting objects: 22% (384/1741) 2024-08-20T22:24:06.7459774Z remote: Counting objects: 23% (401/1741) 2024-08-20T22:24:06.7460313Z remote: Counting objects: 24% (418/1741) 2024-08-20T22:24:06.7460840Z remote: Counting objects: 25% (436/1741) 2024-08-20T22:24:06.7461473Z remote: Counting objects: 26% (453/1741) 2024-08-20T22:24:06.7462583Z remote: Counting objects: 27% (471/1741) 2024-08-20T22:24:06.7463132Z remote: Counting objects: 28% (488/1741) 2024-08-20T22:24:06.7463657Z remote: Counting objects: 29% (505/1741) 2024-08-20T22:24:06.7464188Z remote: Counting objects: 30% (523/1741) 2024-08-20T22:24:06.7464715Z remote: Counting objects: 31% (540/1741) 2024-08-20T22:24:06.7465244Z remote: Counting objects: 32% (558/1741) 2024-08-20T22:24:06.7465768Z remote: Counting objects: 33% (575/1741) 2024-08-20T22:24:06.7466508Z remote: Counting objects: 34% (592/1741) 2024-08-20T22:24:06.7467046Z remote: Counting objects: 35% (610/1741) 2024-08-20T22:24:06.7467568Z remote: Counting objects: 36% (627/1741) 2024-08-20T22:24:06.7468086Z remote: Counting objects: 37% (645/1741) 2024-08-20T22:24:06.7468611Z remote: Counting objects: 38% (662/1741) 2024-08-20T22:24:06.7469132Z remote: Counting objects: 39% (679/1741) 2024-08-20T22:24:06.7469686Z remote: Counting objects: 40% (697/1741) 2024-08-20T22:24:06.7470231Z remote: Counting objects: 41% (714/1741) 2024-08-20T22:24:06.7470758Z remote: Counting objects: 42% (732/1741) 2024-08-20T22:24:06.7471916Z remote: Counting objects: 43% (749/1741) 2024-08-20T22:24:06.7472452Z remote: Counting objects: 44% (767/1741) 2024-08-20T22:24:06.7473002Z remote: Counting objects: 45% (784/1741) 2024-08-20T22:24:06.7473577Z remote: Counting objects: 46% (801/1741) 2024-08-20T22:24:06.7474179Z remote: Counting objects: 47% (819/1741) 2024-08-20T22:24:06.7474810Z remote: Counting objects: 48% (836/1741) 2024-08-20T22:24:06.7475330Z remote: Counting objects: 49% (854/1741) 2024-08-20T22:24:06.7475860Z remote: Counting objects: 50% (871/1741) 2024-08-20T22:24:06.7476395Z remote: Counting objects: 51% (888/1741) 2024-08-20T22:24:06.7476920Z remote: Counting objects: 52% (906/1741) 2024-08-20T22:24:06.7477442Z remote: Counting objects: 53% (923/1741) 2024-08-20T22:24:06.7477971Z remote: Counting objects: 54% (941/1741) 2024-08-20T22:24:06.7478494Z remote: Counting objects: 55% (958/1741) 2024-08-20T22:24:06.7479020Z remote: Counting objects: 56% (975/1741) 2024-08-20T22:24:06.7479539Z remote: Counting objects: 57% (993/1741) 2024-08-20T22:24:06.7480071Z remote: Counting objects: 58% (1010/1741) 2024-08-20T22:24:06.7480617Z remote: Counting objects: 59% (1028/1741) 2024-08-20T22:24:06.7481179Z remote: Counting objects: 60% (1045/1741) 2024-08-20T22:24:06.7481717Z remote: Counting objects: 61% (1063/1741) 2024-08-20T22:24:06.7482257Z remote: Counting objects: 62% (1080/1741) 2024-08-20T22:24:06.7482792Z remote: Counting objects: 63% (1097/1741) 2024-08-20T22:24:06.7483329Z remote: Counting objects: 64% (1115/1741) 2024-08-20T22:24:06.7483856Z remote: Counting objects: 65% (1132/1741) 2024-08-20T22:24:06.7484389Z remote: Counting objects: 66% (1150/1741) 2024-08-20T22:24:06.7484923Z remote: Counting objects: 67% (1167/1741) 2024-08-20T22:24:06.7485458Z remote: Counting objects: 68% (1184/1741) 2024-08-20T22:24:06.7485986Z remote: Counting objects: 69% (1202/1741) 2024-08-20T22:24:06.7486518Z remote: Counting objects: 70% (1219/1741) 2024-08-20T22:24:06.7487049Z remote: Counting objects: 71% (1237/1741) 2024-08-20T22:24:06.7487581Z remote: Counting objects: 72% (1254/1741) 2024-08-20T22:24:06.7488110Z remote: Counting objects: 73% (1271/1741) 2024-08-20T22:24:06.7488642Z remote: Counting objects: 74% (1289/1741) 2024-08-20T22:24:06.7489177Z remote: Counting objects: 75% (1306/1741) 2024-08-20T22:24:06.7489706Z remote: Counting objects: 76% (1324/1741) 2024-08-20T22:24:06.7490169Z remote: Counting objects: 77% (1341/1741) 2024-08-20T22:24:06.7490698Z remote: Counting objects: 78% (1358/1741) 2024-08-20T22:24:06.7491231Z remote: Counting objects: 79% (1376/1741) 2024-08-20T22:24:06.7491754Z remote: Counting objects: 80% (1393/1741) 2024-08-20T22:24:06.7492286Z remote: Counting objects: 81% (1411/1741) 2024-08-20T22:24:06.7492937Z remote: Counting objects: 82% (1428/1741) 2024-08-20T22:24:06.7493465Z remote: Counting objects: 83% (1446/1741) 2024-08-20T22:24:06.7493988Z remote: Counting objects: 84% (1463/1741) 2024-08-20T22:24:06.7494517Z remote: Counting objects: 85% (1480/1741) 2024-08-20T22:24:06.7495046Z remote: Counting objects: 86% (1498/1741) 2024-08-20T22:24:06.7495574Z remote: Counting objects: 87% (1515/1741) 2024-08-20T22:24:06.7496098Z remote: Counting objects: 88% (1533/1741) 2024-08-20T22:24:06.7496713Z remote: Counting objects: 89% (1550/1741) 2024-08-20T22:24:06.7497244Z remote: Counting objects: 90% (1567/1741) 2024-08-20T22:24:06.7497773Z remote: Counting objects: 91% (1585/1741) 2024-08-20T22:24:06.7498296Z remote: Counting objects: 92% (1602/1741) 2024-08-20T22:24:06.7498825Z remote: Counting objects: 93% (1620/1741) 2024-08-20T22:24:06.7499354Z remote: Counting objects: 94% (1637/1741) 2024-08-20T22:24:06.7499889Z remote: Counting objects: 95% (1654/1741) 2024-08-20T22:24:06.7500409Z remote: Counting objects: 96% (1672/1741) 2024-08-20T22:24:06.7500940Z remote: Counting objects: 97% (1689/1741) 2024-08-20T22:24:06.7501472Z remote: Counting objects: 98% (1707/1741) 2024-08-20T22:24:06.7502187Z remote: Counting objects: 99% (1724/1741) 2024-08-20T22:24:06.7502711Z remote: Counting objects: 100% (1741/1741) 2024-08-20T22:24:06.7503278Z remote: Counting objects: 100% (1741/1741), done. 2024-08-20T22:24:06.7503863Z remote: Compressing objects: 0% (1/898) 2024-08-20T22:24:06.7553559Z remote: Compressing objects: 1% (9/898) 2024-08-20T22:24:06.7675414Z remote: Compressing objects: 2% (18/898) 2024-08-20T22:24:06.7762981Z remote: Compressing objects: 3% (27/898) 2024-08-20T22:24:06.7816137Z remote: Compressing objects: 4% (36/898) 2024-08-20T22:24:06.7867856Z remote: Compressing objects: 5% (45/898) 2024-08-20T22:24:06.7902098Z remote: Compressing objects: 6% (54/898) 2024-08-20T22:24:06.7928114Z remote: Compressing objects: 7% (63/898) 2024-08-20T22:24:06.7966730Z remote: Compressing objects: 8% (72/898) 2024-08-20T22:24:06.7999516Z remote: Compressing objects: 9% (81/898) 2024-08-20T22:24:06.8028790Z remote: Compressing objects: 10% (90/898) 2024-08-20T22:24:06.8042286Z remote: Compressing objects: 11% (99/898) 2024-08-20T22:24:06.8054293Z remote: Compressing objects: 12% (108/898) 2024-08-20T22:24:06.8070174Z remote: Compressing objects: 13% (117/898) 2024-08-20T22:24:06.8086098Z remote: Compressing objects: 14% (126/898) 2024-08-20T22:24:06.8095646Z remote: Compressing objects: 15% (135/898) 2024-08-20T22:24:06.8104428Z remote: Compressing objects: 16% (144/898) 2024-08-20T22:24:06.8110642Z remote: Compressing objects: 17% (153/898) 2024-08-20T22:24:06.8113910Z remote: Compressing objects: 18% (162/898) 2024-08-20T22:24:06.8118564Z remote: Compressing objects: 19% (171/898) 2024-08-20T22:24:06.8119711Z remote: Compressing objects: 20% (180/898) 2024-08-20T22:24:06.8120526Z remote: Compressing objects: 21% (189/898) 2024-08-20T22:24:06.8121816Z remote: Compressing objects: 22% (198/898) 2024-08-20T22:24:06.8123981Z remote: Compressing objects: 23% (207/898) 2024-08-20T22:24:06.8125008Z remote: Compressing objects: 24% (216/898) 2024-08-20T22:24:06.8126147Z remote: Compressing objects: 25% (225/898) 2024-08-20T22:24:06.8130571Z remote: Compressing objects: 26% (234/898) 2024-08-20T22:24:06.8137458Z remote: Compressing objects: 27% (243/898) 2024-08-20T22:24:06.8140605Z remote: Compressing objects: 28% (252/898) 2024-08-20T22:24:06.8146383Z remote: Compressing objects: 29% (261/898) 2024-08-20T22:24:06.8149891Z remote: Compressing objects: 30% (270/898) 2024-08-20T22:24:06.8152159Z remote: Compressing objects: 31% (279/898) 2024-08-20T22:24:06.8153380Z remote: Compressing objects: 32% (288/898) 2024-08-20T22:24:06.8157996Z remote: Compressing objects: 33% (297/898) 2024-08-20T22:24:06.8160673Z remote: Compressing objects: 34% (306/898) 2024-08-20T22:24:06.8163214Z remote: Compressing objects: 35% (315/898) 2024-08-20T22:24:06.8165683Z remote: Compressing objects: 36% (324/898) 2024-08-20T22:24:06.8167852Z remote: Compressing objects: 37% (333/898) 2024-08-20T22:24:06.8169522Z remote: Compressing objects: 38% (342/898) 2024-08-20T22:24:06.8171437Z remote: Compressing objects: 39% (351/898) 2024-08-20T22:24:06.8173408Z remote: Compressing objects: 40% (360/898) 2024-08-20T22:24:06.8177112Z remote: Compressing objects: 41% (369/898) 2024-08-20T22:24:06.8178088Z remote: Compressing objects: 42% (378/898) 2024-08-20T22:24:06.8178659Z remote: Compressing objects: 43% (387/898) 2024-08-20T22:24:06.8180022Z remote: Compressing objects: 44% (396/898) 2024-08-20T22:24:06.8181033Z remote: Compressing objects: 45% (405/898) 2024-08-20T22:24:06.8181979Z remote: Compressing objects: 46% (414/898) 2024-08-20T22:24:06.8182554Z remote: Compressing objects: 47% (423/898) 2024-08-20T22:24:06.8183294Z remote: Compressing objects: 48% (432/898) 2024-08-20T22:24:06.8184006Z remote: Compressing objects: 49% (441/898) 2024-08-20T22:24:06.8184817Z remote: Compressing objects: 50% (449/898) 2024-08-20T22:24:06.8185369Z remote: Compressing objects: 51% (458/898) 2024-08-20T22:24:06.8186143Z remote: Compressing objects: 52% (467/898) 2024-08-20T22:24:06.8186686Z remote: Compressing objects: 53% (476/898) 2024-08-20T22:24:06.8187235Z remote: Compressing objects: 54% (485/898) 2024-08-20T22:24:06.8187783Z remote: Compressing objects: 55% (494/898) 2024-08-20T22:24:06.8188326Z remote: Compressing objects: 56% (503/898) 2024-08-20T22:24:06.8188872Z remote: Compressing objects: 57% (512/898) 2024-08-20T22:24:06.8189407Z remote: Compressing objects: 58% (521/898) 2024-08-20T22:24:06.8190181Z remote: Compressing objects: 59% (530/898) 2024-08-20T22:24:06.8190724Z remote: Compressing objects: 60% (539/898) 2024-08-20T22:24:06.8191272Z remote: Compressing objects: 61% (548/898) 2024-08-20T22:24:06.8191808Z remote: Compressing objects: 62% (557/898) 2024-08-20T22:24:06.8192348Z remote: Compressing objects: 63% (566/898) 2024-08-20T22:24:06.8192892Z remote: Compressing objects: 64% (575/898) 2024-08-20T22:24:06.8193434Z remote: Compressing objects: 65% (584/898) 2024-08-20T22:24:06.8193971Z remote: Compressing objects: 66% (593/898) 2024-08-20T22:24:06.8194513Z remote: Compressing objects: 67% (602/898) 2024-08-20T22:24:06.8195060Z remote: Compressing objects: 68% (611/898) 2024-08-20T22:24:06.8195595Z remote: Compressing objects: 69% (620/898) 2024-08-20T22:24:06.8196135Z remote: Compressing objects: 70% (629/898) 2024-08-20T22:24:06.8196686Z remote: Compressing objects: 71% (638/898) 2024-08-20T22:24:06.8197226Z remote: Compressing objects: 72% (647/898) 2024-08-20T22:24:06.8197760Z remote: Compressing objects: 73% (656/898) 2024-08-20T22:24:06.8198305Z remote: Compressing objects: 74% (665/898) 2024-08-20T22:24:06.8199073Z remote: Compressing objects: 75% (674/898) 2024-08-20T22:24:06.8201306Z remote: Compressing objects: 76% (683/898) 2024-08-20T22:24:06.8209319Z remote: Compressing objects: 77% (692/898) 2024-08-20T22:24:06.8214025Z remote: Compressing objects: 78% (701/898) 2024-08-20T22:24:06.8219997Z remote: Compressing objects: 79% (710/898) 2024-08-20T22:24:06.8227000Z remote: Compressing objects: 80% (719/898) 2024-08-20T22:24:06.8236830Z remote: Compressing objects: 81% (728/898) 2024-08-20T22:24:06.8243199Z remote: Compressing objects: 82% (737/898) 2024-08-20T22:24:06.8247277Z remote: Compressing objects: 83% (746/898) 2024-08-20T22:24:06.8251415Z remote: Compressing objects: 84% (755/898) 2024-08-20T22:24:06.8254652Z remote: Compressing objects: 85% (764/898) 2024-08-20T22:24:06.8259621Z remote: Compressing objects: 86% (773/898) 2024-08-20T22:24:06.8264742Z remote: Compressing objects: 87% (782/898) 2024-08-20T22:24:06.8267968Z remote: Compressing objects: 88% (791/898) 2024-08-20T22:24:06.8271440Z remote: Compressing objects: 89% (800/898) 2024-08-20T22:24:06.8274593Z remote: Compressing objects: 90% (809/898) 2024-08-20T22:24:06.8277740Z remote: Compressing objects: 91% (818/898) 2024-08-20T22:24:06.8281297Z remote: Compressing objects: 92% (827/898) 2024-08-20T22:24:06.8284989Z remote: Compressing objects: 93% (836/898) 2024-08-20T22:24:06.8287570Z remote: Compressing objects: 94% (845/898) 2024-08-20T22:24:06.8290256Z remote: Compressing objects: 95% (854/898) 2024-08-20T22:24:06.8293036Z remote: Compressing objects: 96% (863/898) 2024-08-20T22:24:06.8296189Z remote: Compressing objects: 97% (872/898) 2024-08-20T22:24:06.8298311Z remote: Compressing objects: 98% (881/898) 2024-08-20T22:24:06.8298866Z remote: Compressing objects: 99% (890/898) 2024-08-20T22:24:06.8299409Z remote: Compressing objects: 100% (898/898) 2024-08-20T22:24:06.8300009Z remote: Compressing objects: 100% (898/898), done. 2024-08-20T22:24:06.8380398Z Receiving objects: 0% (1/31001) 2024-08-20T22:24:06.8422559Z Receiving objects: 1% (311/31001) 2024-08-20T22:24:06.8462387Z Receiving objects: 2% (621/31001) 2024-08-20T22:24:06.8498038Z Receiving objects: 3% (931/31001) 2024-08-20T22:24:06.8531351Z Receiving objects: 4% (1241/31001) 2024-08-20T22:24:06.8565045Z Receiving objects: 5% (1551/31001) 2024-08-20T22:24:06.8602006Z Receiving objects: 6% (1861/31001) 2024-08-20T22:24:06.8640971Z Receiving objects: 7% (2171/31001) 2024-08-20T22:24:06.8671521Z Receiving objects: 8% (2481/31001) 2024-08-20T22:24:06.8704179Z Receiving objects: 9% (2791/31001) 2024-08-20T22:24:06.8774816Z Receiving objects: 10% (3101/31001) 2024-08-20T22:24:06.8823058Z Receiving objects: 11% (3411/31001) 2024-08-20T22:24:06.8889968Z Receiving objects: 12% (3721/31001) 2024-08-20T22:24:06.9021572Z Receiving objects: 13% (4031/31001) 2024-08-20T22:24:06.9165598Z Receiving objects: 14% (4341/31001) 2024-08-20T22:24:06.9237184Z Receiving objects: 15% (4651/31001) 2024-08-20T22:24:06.9310487Z Receiving objects: 16% (4961/31001) 2024-08-20T22:24:06.9369851Z Receiving objects: 17% (5271/31001) 2024-08-20T22:24:06.9397796Z Receiving objects: 18% (5581/31001) 2024-08-20T22:24:06.9456147Z Receiving objects: 19% (5891/31001) 2024-08-20T22:24:06.9488833Z Receiving objects: 20% (6201/31001) 2024-08-20T22:24:06.9504562Z Receiving objects: 21% (6511/31001) 2024-08-20T22:24:06.9672466Z Receiving objects: 22% (6821/31001) 2024-08-20T22:24:07.8328475Z Receiving objects: 23% (7131/31001) 2024-08-20T22:24:07.9471091Z Receiving objects: 23% (7377/31001), 75.89 MiB | 75.89 MiB/s 2024-08-20T22:24:08.4975224Z Receiving objects: 24% (7441/31001), 75.89 MiB | 75.89 MiB/s 2024-08-20T22:24:08.5694160Z Receiving objects: 25% (7751/31001), 113.50 MiB | 75.66 MiB/s 2024-08-20T22:24:08.6421461Z Receiving objects: 26% (8061/31001), 113.50 MiB | 75.66 MiB/s 2024-08-20T22:24:08.7133965Z Receiving objects: 27% (8371/31001), 113.50 MiB | 75.66 MiB/s 2024-08-20T22:24:08.7844642Z Receiving objects: 28% (8681/31001), 113.50 MiB | 75.66 MiB/s 2024-08-20T22:24:08.8325440Z Receiving objects: 29% (8991/31001), 113.50 MiB | 75.66 MiB/s 2024-08-20T22:24:08.8564491Z Receiving objects: 29% (9199/31001), 113.50 MiB | 75.66 MiB/s 2024-08-20T22:24:08.9279937Z Receiving objects: 30% (9301/31001), 151.88 MiB | 75.94 MiB/s 2024-08-20T22:24:08.9995565Z Receiving objects: 31% (9611/31001), 151.88 MiB | 75.94 MiB/s 2024-08-20T22:24:09.2331825Z Receiving objects: 32% (9921/31001), 151.88 MiB | 75.94 MiB/s 2024-08-20T22:24:09.3073613Z Receiving objects: 33% (10231/31001), 151.88 MiB | 75.94 MiB/s 2024-08-20T22:24:09.3197136Z Receiving objects: 34% (10541/31001), 151.88 MiB | 75.94 MiB/s 2024-08-20T22:24:09.5792685Z Receiving objects: 35% (10851/31001), 151.88 MiB | 75.94 MiB/s 2024-08-20T22:24:09.8272536Z Receiving objects: 36% (11161/31001), 184.95 MiB | 73.98 MiB/s 2024-08-20T22:24:09.8289709Z Receiving objects: 37% (11471/31001), 184.95 MiB | 73.98 MiB/s 2024-08-20T22:24:09.8312046Z Receiving objects: 38% (11781/31001), 184.95 MiB | 73.98 MiB/s 2024-08-20T22:24:09.8325004Z Receiving objects: 39% (12091/31001), 184.95 MiB | 73.98 MiB/s 2024-08-20T22:24:09.8330503Z Receiving objects: 39% (12283/31001), 184.95 MiB | 73.98 MiB/s 2024-08-20T22:24:09.8344973Z Receiving objects: 40% (12401/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8356791Z Receiving objects: 41% (12711/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8369477Z Receiving objects: 42% (13021/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8385449Z Receiving objects: 43% (13331/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8401469Z Receiving objects: 44% (13641/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8409459Z Receiving objects: 45% (13951/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8414398Z Receiving objects: 46% (14261/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8428710Z Receiving objects: 47% (14571/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8450105Z Receiving objects: 48% (14881/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8463281Z Receiving objects: 49% (15191/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8663804Z Receiving objects: 50% (15501/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8669940Z Receiving objects: 51% (15811/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8677208Z Receiving objects: 52% (16121/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8683395Z Receiving objects: 53% (16431/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8688862Z Receiving objects: 54% (16741/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8705738Z Receiving objects: 55% (17051/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8716202Z Receiving objects: 56% (17361/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8722434Z Receiving objects: 57% (17671/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8735952Z Receiving objects: 58% (17981/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8747989Z Receiving objects: 59% (18291/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8758859Z Receiving objects: 60% (18601/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8896005Z Receiving objects: 61% (18911/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8901606Z Receiving objects: 62% (19221/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8907666Z Receiving objects: 63% (19531/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8915684Z Receiving objects: 64% (19841/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8923256Z Receiving objects: 65% (20151/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.8931907Z Receiving objects: 66% (20461/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9189065Z Receiving objects: 67% (20771/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9587032Z Receiving objects: 68% (21081/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9595985Z Receiving objects: 69% (21391/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9609657Z Receiving objects: 70% (21701/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9620782Z Receiving objects: 71% (22011/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9635379Z Receiving objects: 72% (22321/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9644501Z Receiving objects: 73% (22631/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9660626Z Receiving objects: 74% (22941/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9671822Z Receiving objects: 75% (23251/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9679085Z Receiving objects: 76% (23561/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9687347Z Receiving objects: 77% (23871/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9900459Z Receiving objects: 78% (24181/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9933686Z Receiving objects: 79% (24491/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9943085Z Receiving objects: 80% (24801/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9984048Z Receiving objects: 81% (25111/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:09.9984783Z Receiving objects: 82% (25421/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0016922Z Receiving objects: 83% (25731/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0155809Z Receiving objects: 84% (26041/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0160851Z Receiving objects: 85% (26351/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0168876Z Receiving objects: 86% (26661/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0173960Z Receiving objects: 87% (26971/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0181165Z Receiving objects: 88% (27281/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0187374Z Receiving objects: 89% (27591/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0298717Z Receiving objects: 90% (27901/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0312964Z Receiving objects: 91% (28211/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0324856Z Receiving objects: 92% (28521/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0341798Z Receiving objects: 93% (28831/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0379195Z Receiving objects: 94% (29141/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0391520Z Receiving objects: 95% (29451/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0399912Z Receiving objects: 96% (29761/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0407514Z Receiving objects: 97% (30071/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0480675Z Receiving objects: 98% (30381/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0506945Z Receiving objects: 99% (30691/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0507973Z remote: Total 31001 (delta 1013), reused 1477 (delta 824), pack-reused 29260 (from 1) 2024-08-20T22:24:10.0525470Z Receiving objects: 100% (31001/31001), 222.05 MiB | 74.02 MiB/s 2024-08-20T22:24:10.0526177Z Receiving objects: 100% (31001/31001), 232.50 MiB | 72.21 MiB/s, done. 2024-08-20T22:24:10.0557710Z Resolving deltas: 0% (0/16733) 2024-08-20T22:24:10.0571479Z Resolving deltas: 1% (168/16733) 2024-08-20T22:24:10.0583502Z Resolving deltas: 2% (390/16733) 2024-08-20T22:24:10.0607058Z Resolving deltas: 3% (508/16733) 2024-08-20T22:24:10.0620840Z Resolving deltas: 4% (670/16733) 2024-08-20T22:24:10.0629949Z Resolving deltas: 5% (885/16733) 2024-08-20T22:24:10.0643453Z Resolving deltas: 6% (1006/16733) 2024-08-20T22:24:10.0652722Z Resolving deltas: 7% (1178/16733) 2024-08-20T22:24:10.0661630Z Resolving deltas: 8% (1362/16733) 2024-08-20T22:24:10.0672316Z Resolving deltas: 9% (1519/16733) 2024-08-20T22:24:10.0682424Z Resolving deltas: 10% (1684/16733) 2024-08-20T22:24:10.0695462Z Resolving deltas: 11% (1841/16733) 2024-08-20T22:24:10.0707981Z Resolving deltas: 12% (2008/16733) 2024-08-20T22:24:10.0720848Z Resolving deltas: 13% (2183/16733) 2024-08-20T22:24:10.0735575Z Resolving deltas: 14% (2391/16733) 2024-08-20T22:24:10.0752751Z Resolving deltas: 15% (2511/16733) 2024-08-20T22:24:10.0766561Z Resolving deltas: 16% (2688/16733) 2024-08-20T22:24:10.0803340Z Resolving deltas: 17% (2858/16733) 2024-08-20T22:24:10.0830139Z Resolving deltas: 18% (3032/16733) 2024-08-20T22:24:10.0849190Z Resolving deltas: 19% (3189/16733) 2024-08-20T22:24:10.0851176Z Resolving deltas: 20% (3493/16733) 2024-08-20T22:24:10.0860746Z Resolving deltas: 21% (3520/16733) 2024-08-20T22:24:10.0876702Z Resolving deltas: 22% (3687/16733) 2024-08-20T22:24:10.0881854Z Resolving deltas: 23% (3951/16733) 2024-08-20T22:24:10.0896317Z Resolving deltas: 24% (4043/16733) 2024-08-20T22:24:10.0930152Z Resolving deltas: 25% (4192/16733) 2024-08-20T22:24:10.0950587Z Resolving deltas: 26% (4351/16733) 2024-08-20T22:24:10.0963194Z Resolving deltas: 27% (4540/16733) 2024-08-20T22:24:10.0978880Z Resolving deltas: 28% (4686/16733) 2024-08-20T22:24:10.0995996Z Resolving deltas: 29% (4855/16733) 2024-08-20T22:24:10.1009495Z Resolving deltas: 30% (5021/16733) 2024-08-20T22:24:10.1021083Z Resolving deltas: 31% (5188/16733) 2024-08-20T22:24:10.1032356Z Resolving deltas: 32% (5385/16733) 2024-08-20T22:24:10.1043564Z Resolving deltas: 33% (5558/16733) 2024-08-20T22:24:10.1065019Z Resolving deltas: 34% (5696/16733) 2024-08-20T22:24:10.1083162Z Resolving deltas: 35% (5859/16733) 2024-08-20T22:24:10.1093847Z Resolving deltas: 36% (6040/16733) 2024-08-20T22:24:10.1116647Z Resolving deltas: 37% (6211/16733) 2024-08-20T22:24:10.1127711Z Resolving deltas: 38% (6385/16733) 2024-08-20T22:24:10.1136023Z Resolving deltas: 39% (6572/16733) 2024-08-20T22:24:10.1159580Z Resolving deltas: 40% (6721/16733) 2024-08-20T22:24:10.1164940Z Resolving deltas: 41% (6920/16733) 2024-08-20T22:24:10.1173423Z Resolving deltas: 42% (7038/16733) 2024-08-20T22:24:10.1185038Z Resolving deltas: 43% (7199/16733) 2024-08-20T22:24:10.1208668Z Resolving deltas: 44% (7366/16733) 2024-08-20T22:24:10.1225497Z Resolving deltas: 45% (7531/16733) 2024-08-20T22:24:10.1237389Z Resolving deltas: 46% (7700/16733) 2024-08-20T22:24:10.1257810Z Resolving deltas: 47% (7874/16733) 2024-08-20T22:24:10.1269590Z Resolving deltas: 48% (8038/16733) 2024-08-20T22:24:10.1276013Z Resolving deltas: 49% (8229/16733) 2024-08-20T22:24:10.1287073Z Resolving deltas: 50% (8384/16733) 2024-08-20T22:24:10.1307472Z Resolving deltas: 51% (8543/16733) 2024-08-20T22:24:10.1313658Z Resolving deltas: 52% (8784/16733) 2024-08-20T22:24:10.1319603Z Resolving deltas: 53% (8907/16733) 2024-08-20T22:24:10.1332588Z Resolving deltas: 54% (9036/16733) 2024-08-20T22:24:10.1344334Z Resolving deltas: 55% (9204/16733) 2024-08-20T22:24:10.1351994Z Resolving deltas: 56% (9374/16733) 2024-08-20T22:24:10.1362424Z Resolving deltas: 57% (9546/16733) 2024-08-20T22:24:10.1367166Z Resolving deltas: 58% (9776/16733) 2024-08-20T22:24:10.1378297Z Resolving deltas: 59% (9873/16733) 2024-08-20T22:24:10.1390410Z Resolving deltas: 60% (10046/16733) 2024-08-20T22:24:10.1405106Z Resolving deltas: 61% (10258/16733) 2024-08-20T22:24:10.1417325Z Resolving deltas: 62% (10428/16733) 2024-08-20T22:24:10.1429818Z Resolving deltas: 63% (10631/16733) 2024-08-20T22:24:10.1451671Z Resolving deltas: 64% (10811/16733) 2024-08-20T22:24:10.1463488Z Resolving deltas: 66% (11085/16733) 2024-08-20T22:24:10.1485212Z Resolving deltas: 67% (11231/16733) 2024-08-20T22:24:10.1492243Z Resolving deltas: 68% (11387/16733) 2024-08-20T22:24:10.1505326Z Resolving deltas: 69% (11546/16733) 2024-08-20T22:24:10.1521388Z Resolving deltas: 70% (11715/16733) 2024-08-20T22:24:10.1534339Z Resolving deltas: 71% (11882/16733) 2024-08-20T22:24:10.1545146Z Resolving deltas: 72% (12048/16733) 2024-08-20T22:24:10.1556709Z Resolving deltas: 73% (12217/16733) 2024-08-20T22:24:10.1567102Z Resolving deltas: 74% (12386/16733) 2024-08-20T22:24:10.1585925Z Resolving deltas: 75% (12562/16733) 2024-08-20T22:24:10.1591162Z Resolving deltas: 76% (12799/16733) 2024-08-20T22:24:10.1599812Z Resolving deltas: 77% (12891/16733) 2024-08-20T22:24:10.1614442Z Resolving deltas: 78% (13054/16733) 2024-08-20T22:24:10.1624100Z Resolving deltas: 79% (13230/16733) 2024-08-20T22:24:10.1638691Z Resolving deltas: 80% (13387/16733) 2024-08-20T22:24:10.1649183Z Resolving deltas: 81% (13582/16733) 2024-08-20T22:24:10.1661965Z Resolving deltas: 82% (13745/16733) 2024-08-20T22:24:10.1673480Z Resolving deltas: 83% (13898/16733) 2024-08-20T22:24:10.1681159Z Resolving deltas: 84% (14104/16733) 2024-08-20T22:24:10.1689598Z Resolving deltas: 85% (14241/16733) 2024-08-20T22:24:10.1699628Z Resolving deltas: 86% (14409/16733) 2024-08-20T22:24:10.1709032Z Resolving deltas: 87% (14559/16733) 2024-08-20T22:24:10.1720293Z Resolving deltas: 88% (14730/16733) 2024-08-20T22:24:10.1728083Z Resolving deltas: 89% (14905/16733) 2024-08-20T22:24:10.1738834Z Resolving deltas: 90% (15072/16733) 2024-08-20T22:24:10.1749942Z Resolving deltas: 91% (15229/16733) 2024-08-20T22:24:10.1756363Z Resolving deltas: 92% (15396/16733) 2024-08-20T22:24:10.1764383Z Resolving deltas: 93% (15563/16733) 2024-08-20T22:24:10.1775508Z Resolving deltas: 94% (15738/16733) 2024-08-20T22:24:10.1785884Z Resolving deltas: 95% (15981/16733) 2024-08-20T22:24:10.1791384Z Resolving deltas: 96% (16135/16733) 2024-08-20T22:24:10.1798752Z Resolving deltas: 97% (16246/16733) 2024-08-20T22:24:10.1854370Z Resolving deltas: 98% (16400/16733) 2024-08-20T22:24:10.1915109Z Resolving deltas: 99% (16586/16733) 2024-08-20T22:24:10.1915537Z Resolving deltas: 100% (16733/16733) 2024-08-20T22:24:10.1915986Z Resolving deltas: 100% (16733/16733), done. 2024-08-20T22:24:11.1920805Z + pushd torchbench 2024-08-20T22:24:11.1921250Z ~/workspace/torchbench ~/workspace 2024-08-20T22:24:11.1921770Z + git checkout 23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T22:24:11.2708317Z Note: switching to '23512dbebd44a11eb84afbf53c3c071dd105297e'. 2024-08-20T22:24:11.2708872Z 2024-08-20T22:24:11.2709426Z You are in 'detached HEAD' state. You can look around, make experimental 2024-08-20T22:24:11.2710494Z changes and commit them, and you can discard any commits you make in this 2024-08-20T22:24:11.2711382Z state without impacting any branches by switching back to a branch. 2024-08-20T22:24:11.2711829Z 2024-08-20T22:24:11.2712142Z If you want to create a new branch to retain commits you create, you may 2024-08-20T22:24:11.2712962Z do so (now or later) by using -c with the switch command. Example: 2024-08-20T22:24:11.2713386Z 2024-08-20T22:24:11.2713594Z git switch -c 2024-08-20T22:24:11.2713866Z 2024-08-20T22:24:11.2714004Z Or undo this operation with: 2024-08-20T22:24:11.2714264Z 2024-08-20T22:24:11.2714382Z git switch - 2024-08-20T22:24:11.2714572Z 2024-08-20T22:24:11.2714907Z Turn off this advice by setting config variable advice.detachedHead to false 2024-08-20T22:24:11.2715396Z 2024-08-20T22:24:11.2715567Z HEAD is now at 23512dbe Pin numpy version 2024-08-20T22:24:11.2719377Z + '[' '' ']' 2024-08-20T22:24:11.2719898Z + python install.py --continue_on_fail 2024-08-20T22:24:13.9166508Z checking packages torch, torchvision, torchaudio are installed...OK 2024-08-20T22:24:32.0183749Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch...OK 2024-08-20T22:24:35.8200665Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Background_Matting...OK 2024-08-20T22:24:37.7226527Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/LearningToPaint...OK 2024-08-20T22:24:39.3070153Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Super_SloMo...OK 2024-08-20T22:24:39.3307880Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/alexnet...OK 2024-08-20T22:24:46.9432434Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_edgecnn...OK 2024-08-20T22:24:49.6645285Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gcn...OK 2024-08-20T22:24:52.3833316Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gin...OK 2024-08-20T22:24:55.0950424Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_sage...OK 2024-08-20T22:24:55.0953604Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/cm3leon_generate...SKIP - No install.py is found 2024-08-20T22:24:56.1457376Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dcgan...OK 2024-08-20T22:24:58.8084604Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/demucs...OK 2024-08-20T22:24:58.8321444Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/densenet121...OK 2024-08-20T22:25:55.7810657Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_c4...OK 2024-08-20T22:26:04.2841234Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_dc5...OK 2024-08-20T22:26:11.4616803Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_fpn...OK 2024-08-20T22:26:18.1253713Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_c4...OK 2024-08-20T22:26:26.3147458Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_dc5...OK 2024-08-20T22:26:33.2971843Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_fpn...OK 2024-08-20T22:26:39.6635386Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fcos_r_50_fpn...OK 2024-08-20T22:26:46.4304939Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn...OK 2024-08-20T22:26:53.4409394Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_c4...OK 2024-08-20T22:27:00.4974139Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_fpn...OK 2024-08-20T22:27:07.2863327Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_c4...OK 2024-08-20T22:27:14.1972713Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_fpn...OK 2024-08-20T22:27:15.4013236Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dlrm...OK 2024-08-20T22:27:51.3503200Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_det_predictor...OK 2024-08-20T22:27:55.8638339Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_reco_predictor...OK 2024-08-20T22:28:01.0665935Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/drq...OK 2024-08-20T22:28:12.1336394Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/fastNLP_Bert...OK 2024-08-20T22:28:13.3228704Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_dp_cifar10...OK 2024-08-20T22:28:14.8005365Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_maml_omniglot...OK 2024-08-20T22:28:21.3577685Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Albert...OK 2024-08-20T22:28:28.7533667Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bart...OK 2024-08-20T22:28:34.2593865Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert...OK 2024-08-20T22:28:41.9184300Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert_large...OK 2024-08-20T22:28:47.7826822Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_BigBird...OK 2024-08-20T22:28:52.7707920Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_DistilBert...OK 2024-08-20T22:28:59.0170419Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2...OK 2024-08-20T22:29:15.4472408Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2_large...OK 2024-08-20T22:29:21.6544438Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Longformer...OK 2024-08-20T22:29:25.5676617Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Reformer...OK 2024-08-20T22:29:30.3070805Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5...OK 2024-08-20T22:29:37.0731653Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_base...OK 2024-08-20T22:29:37.0735273Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_generate...SKIP - No install.py is found 2024-08-20T22:29:50.1013085Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_large...OK 2024-08-20T22:29:54.0387782Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Whisper...OK 2024-08-20T22:29:54.0390424Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_clip...SKIP - No install.py is found 2024-08-20T22:30:01.6278416Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_distil_whisper...OK 2024-08-20T22:30:02.8570805Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/lennard_jones...OK 2024-08-20T22:30:04.1021364Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama...OK 2024-08-20T22:30:48.7931709Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama_v2_7b_16h...OK 2024-08-20T22:32:17.8612601Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llava...OK 2024-08-20T22:32:17.8856026Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml...OK 2024-08-20T22:32:19.3929483Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml_omniglot...OK 2024-08-20T22:32:19.4174458Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mnasnet1_0...OK 2024-08-20T22:32:19.4415380Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2...OK 2024-08-20T22:32:19.4661971Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2_quantized_qat...OK 2024-08-20T22:32:19.4900045Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v3_large...OK 2024-08-20T22:32:19.5145685Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco...OK 2024-08-20T22:32:40.5451737Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moondream...OK 2024-08-20T22:32:40.5454571Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nanogpt...SKIP - No install.py is found 2024-08-20T22:32:41.7911566Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nvidia_deeprecommender...OK 2024-08-20T22:32:44.1991802Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/opacus_cifar10...OK 2024-08-20T22:32:44.2233620Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_densenet...OK 2024-08-20T22:32:44.2473747Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_resnet...OK 2024-08-20T22:32:44.2719075Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_equation_of_state...OK 2024-08-20T22:32:44.2964421Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_isoneutral_mixing...OK 2024-08-20T22:32:44.3210917Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_turbulent_kinetic_energy...OK 2024-08-20T22:32:50.9264149Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_CycleGAN_and_pix2pix...OK 2024-08-20T22:32:52.3844731Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_stargan...OK 2024-08-20T22:32:55.8513295Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_unet...OK 2024-08-20T22:32:55.8759169Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet152...OK 2024-08-20T22:32:55.9000254Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet18...OK 2024-08-20T22:32:55.9241039Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50...OK 2024-08-20T22:32:55.9480143Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50_quantized_qat...OK 2024-08-20T22:32:55.9720472Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnext50_32x4d...OK 2024-08-20T22:33:19.6611555Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam...OK 2024-08-20T22:33:36.5606097Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam_fast...OK 2024-08-20T22:33:36.5850711Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/shufflenet_v2_x1_0...OK 2024-08-20T22:33:36.5852431Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt...SKIP - No install.py is found 2024-08-20T22:33:36.5855257Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt_tp_manual...SKIP - No install.py is found 2024-08-20T22:33:39.1491611Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/soft_actor_critic...OK 2024-08-20T22:33:40.7563485Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/speech_transformer...OK 2024-08-20T22:33:40.7809614Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/squeezenet1_1...OK 2024-08-20T22:33:59.2012125Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_text_encoder...OK 2024-08-20T22:34:05.7942799Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_unet...OK 2024-08-20T22:34:10.6471418Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tacotron2...OK 2024-08-20T22:34:17.6956422Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientdet...OK 2024-08-20T22:34:17.7202715Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientnet...OK 2024-08-20T22:34:17.7447615Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_nfnet...OK 2024-08-20T22:34:17.7689498Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_regnet...OK 2024-08-20T22:34:17.7935336Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_resnest...OK 2024-08-20T22:34:17.8177362Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer...OK 2024-08-20T22:34:17.8420699Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer_large...OK 2024-08-20T22:34:17.8662756Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vovnet...OK 2024-08-20T22:34:27.0335640Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/torch_multimodal_clip...OK 2024-08-20T22:34:39.0737906Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tts_angular...OK 2024-08-20T22:34:39.0995892Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vgg16...OK 2024-08-20T22:34:41.2091540Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vision_maskrcnn...OK 2024-08-20T22:34:43.1734279Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/yolov3...OK 2024-08-20T22:34:43.6569997Z + echo 'Print all dependencies after TorchBench is installed' 2024-08-20T22:34:43.6570681Z Print all dependencies after TorchBench is installed 2024-08-20T22:34:43.6573188Z + python -mpip freeze 2024-08-20T22:34:44.1764002Z absl-py==2.1.0 2024-08-20T22:34:44.1764500Z accelerate==0.33.0 2024-08-20T22:34:44.1764956Z aiohappyeyeballs==2.4.0 2024-08-20T22:34:44.1765436Z aiohttp==3.10.5 2024-08-20T22:34:44.1765820Z aiosignal==1.3.1 2024-08-20T22:34:44.1766341Z annotated-types==0.7.0 2024-08-20T22:34:44.1766889Z antlr4-python3-runtime==4.9.3 2024-08-20T22:34:44.1767359Z anyascii==0.3.2 2024-08-20T22:34:44.1767661Z astroid==3.2.4 2024-08-20T22:34:44.1767955Z astunparse==1.6.3 2024-08-20T22:34:44.1768388Z async-timeout==4.0.3 2024-08-20T22:34:44.1768854Z attrs==23.1.0 2024-08-20T22:34:44.1769164Z audioread==3.0.1 2024-08-20T22:34:44.1769546Z babel==2.16.0 2024-08-20T22:34:44.1769863Z beautifulsoup4==4.12.3 2024-08-20T22:34:44.1771081Z -e git+https://github.com/pytorch/benchmark@23512dbebd44a11eb84afbf53c3c071dd105297e#egg=bert_pytorch&subdirectory=torchbenchmark/models/BERT_pytorch 2024-08-20T22:34:44.1772231Z bibtexparser==2.0.0b7 2024-08-20T22:34:44.1773147Z black==24.8.0 2024-08-20T22:34:44.1773539Z blinker==1.8.2 2024-08-20T22:34:44.1773844Z blis==0.7.11 2024-08-20T22:34:44.1774197Z blobfile==2.1.1 2024-08-20T22:34:44.1774505Z bokeh==3.5.1 2024-08-20T22:34:44.1774832Z boto3==1.19.12 2024-08-20T22:34:44.1775235Z botocore==1.22.12 2024-08-20T22:34:44.1775623Z bs4==0.0.2 2024-08-20T22:34:44.1775993Z cachetools==5.5.0 2024-08-20T22:34:44.1776371Z cardboardlint==1.3.1 2024-08-20T22:34:44.1786591Z catalogue==2.0.10 2024-08-20T22:34:44.1787027Z certifi==2024.7.4 2024-08-20T22:34:44.1787423Z cffi==1.17.0 2024-08-20T22:34:44.1789583Z charset-normalizer==3.3.2 2024-08-20T22:34:44.1790061Z click==8.1.7 2024-08-20T22:34:44.1790436Z clldutils==3.22.2 2024-08-20T22:34:44.1790955Z cloudpathlib==0.18.1 2024-08-20T22:34:44.1791394Z cloudpickle==3.0.0 2024-08-20T22:34:44.1791790Z colorama==0.4.6 2024-08-20T22:34:44.1792091Z colorlog==6.8.2 2024-08-20T22:34:44.1792385Z confection==0.1.5 2024-08-20T22:34:44.1792690Z contourpy==1.2.1 2024-08-20T22:34:44.1793004Z coremltools==5.0b5 2024-08-20T22:34:44.1793322Z cryptography==43.0.0 2024-08-20T22:34:44.1793639Z csvw==3.3.0 2024-08-20T22:34:44.1793920Z cycler==0.12.1 2024-08-20T22:34:44.1794209Z cymem==2.0.8 2024-08-20T22:34:44.1794549Z DALL-E==0.1 2024-08-20T22:34:44.1794830Z datasets==2.21.0 2024-08-20T22:34:44.1795130Z decorator==5.1.1 2024-08-20T22:34:44.1795425Z defusedxml==0.7.1 2024-08-20T22:34:44.1795734Z Deprecated==1.2.14 2024-08-20T22:34:44.1796465Z detectron2 @ git+https://github.com/facebookresearch/detectron2.git@1a4df4d954089534daebbd7a0a45f08ece87056e 2024-08-20T22:34:44.1797334Z diffusers==0.20.2 2024-08-20T22:34:44.1797651Z dill==0.3.8 2024-08-20T22:34:44.1797934Z diskcache==5.6.3 2024-08-20T22:34:44.1798239Z distro==1.9.0 2024-08-20T22:34:44.1798530Z dlinfo==1.2.1 2024-08-20T22:34:44.1798876Z docker-pycreds==0.4.0 2024-08-20T22:34:44.1799204Z dominate==2.9.1 2024-08-20T22:34:44.1800023Z effdet @ git+https://github.com/rwightman/efficientdet-pytorch.git@d43c9e34cd62d22b4205831bb735f6dd83b8e881 2024-08-20T22:34:44.1800816Z einops==0.8.0 2024-08-20T22:34:44.1801120Z exceptiongroup==1.2.2 2024-08-20T22:34:44.1801449Z execnet==2.1.1 2024-08-20T22:34:44.1801750Z expecttest==0.1.6 2024-08-20T22:34:44.1802048Z FastNLP==0.6.0 2024-08-20T22:34:44.1802392Z ffmpeg-python==0.2.0 2024-08-20T22:34:44.1802713Z filelock==3.13.1 2024-08-20T22:34:44.1803004Z Flask==3.0.3 2024-08-20T22:34:44.1803300Z flatbuffers==2.0 2024-08-20T22:34:44.1803608Z fonttools==4.53.1 2024-08-20T22:34:44.1803912Z frozenlist==1.4.1 2024-08-20T22:34:44.1804204Z fsspec==2024.6.1 2024-08-20T22:34:44.1804530Z ftfy==6.2.3 2024-08-20T22:34:44.1805172Z functorch @ git+https://github.com/pytorch/functorch.git@b71aa0b4387b86c278132209b99538be48ef4c74 2024-08-20T22:34:44.1805894Z future==1.0.0 2024-08-20T22:34:44.1806191Z fvcore==0.1.5.post20221221 2024-08-20T22:34:44.1806535Z gdown==5.2.0 2024-08-20T22:34:44.1806821Z geojson==2.5.0 2024-08-20T22:34:44.1807106Z ghstack==0.8.0 2024-08-20T22:34:44.1807398Z gitdb==4.0.11 2024-08-20T22:34:44.1807700Z GitPython==3.1.43 2024-08-20T22:34:44.1808055Z google-auth==2.34.0 2024-08-20T22:34:44.1808420Z google-auth-oauthlib==1.0.0 2024-08-20T22:34:44.1808779Z grpcio==1.65.5 2024-08-20T22:34:44.1809069Z gym==0.26.2 2024-08-20T22:34:44.1809377Z gym-notices==0.0.8 2024-08-20T22:34:44.1809683Z h5py==3.11.0 2024-08-20T22:34:44.1809969Z higher==0.2.1 2024-08-20T22:34:44.1810309Z huggingface-hub==0.24.6 2024-08-20T22:34:44.1810672Z hydra-core==1.3.2 2024-08-20T22:34:44.1810981Z hypothesis==5.35.1 2024-08-20T22:34:44.1811284Z idna==3.7 2024-08-20T22:34:44.1811547Z imageio==2.35.1 2024-08-20T22:34:44.1811863Z importlib_metadata==8.4.0 2024-08-20T22:34:44.1812212Z inflect==7.3.1 2024-08-20T22:34:44.1812504Z iniconfig==2.0.0 2024-08-20T22:34:44.1812797Z iopath==0.1.9 2024-08-20T22:34:44.1813087Z isodate==0.6.1 2024-08-20T22:34:44.1813377Z isort==5.13.2 2024-08-20T22:34:44.1813659Z itsdangerous==2.2.0 2024-08-20T22:34:44.1813971Z Jinja2==3.1.4 2024-08-20T22:34:44.1814257Z jmespath==0.10.0 2024-08-20T22:34:44.1814560Z joblib==1.4.2 2024-08-20T22:34:44.1814967Z jsonpatch==1.33 2024-08-20T22:34:44.1815270Z jsonpointer==3.0.0 2024-08-20T22:34:44.1815580Z jsonschema==4.23.0 2024-08-20T22:34:44.1815979Z jsonschema-specifications==2023.12.1 2024-08-20T22:34:44.1816394Z junitparser==2.1.1 2024-08-20T22:34:44.1816741Z kaldi-io==0.9.8 2024-08-20T22:34:44.1817038Z kiwisolver==1.4.5 2024-08-20T22:34:44.1817331Z kornia==0.7.3 2024-08-20T22:34:44.1817618Z kornia_rs==0.1.5 2024-08-20T22:34:44.1817923Z lameenc==1.7.0 2024-08-20T22:34:44.1818215Z langcodes==3.4.0 2024-08-20T22:34:44.1818512Z langdetect==1.0.9 2024-08-20T22:34:44.1818955Z language-tags==1.2.0 2024-08-20T22:34:44.1819284Z language_data==1.2.0 2024-08-20T22:34:44.1819591Z lark==0.12.0 2024-08-20T22:34:44.1819876Z lazy_loader==0.4 2024-08-20T22:34:44.1820174Z libcst==1.4.0 2024-08-20T22:34:44.1820459Z librosa==0.9.2 2024-08-20T22:34:44.1820746Z lintrunner==0.12.5 2024-08-20T22:34:44.1821052Z llvmlite==0.38.1 2024-08-20T22:34:44.1821348Z lxml==4.9.4 2024-08-20T22:34:44.1821661Z marisa-trie==1.2.0 2024-08-20T22:34:44.1822111Z Markdown==3.7 2024-08-20T22:34:44.1822450Z markdown-it-py==3.0.0 2024-08-20T22:34:44.1822780Z MarkupSafe==2.1.5 2024-08-20T22:34:44.1823081Z matplotlib==3.8.4 2024-08-20T22:34:44.1823384Z mccabe==0.7.0 2024-08-20T22:34:44.1823672Z mdurl==0.1.2 2024-08-20T22:34:44.1823988Z ml-dtypes==0.4.0 2024-08-20T22:34:44.1824296Z MonkeyType==23.3.0 2024-08-20T22:34:44.1824653Z more-itertools==10.4.0 2024-08-20T22:34:44.1824980Z mpmath==1.3.0 2024-08-20T22:34:44.1825261Z msgpack==1.0.8 2024-08-20T22:34:44.1825556Z multidict==6.0.5 2024-08-20T22:34:44.1825874Z multiprocess==0.70.16 2024-08-20T22:34:44.1826197Z murmurhash==1.0.10 2024-08-20T22:34:44.1826502Z musdb==0.4.2 2024-08-20T22:34:44.1826789Z museval==0.4.1 2024-08-20T22:34:44.1827077Z mypy==1.10.0 2024-08-20T22:34:44.1827406Z mypy-extensions==1.0.0 2024-08-20T22:34:44.1827736Z networkx==2.8.8 2024-08-20T22:34:44.1828039Z ninja==1.10.2 2024-08-20T22:34:44.1828316Z nose==1.3.7 2024-08-20T22:34:44.1828590Z numba==0.55.2 2024-08-20T22:34:44.1828872Z numpy==1.21.2 2024-08-20T22:34:44.1829229Z nvidia-cublas-cu12==12.1.3.1 2024-08-20T22:34:44.1829653Z nvidia-cuda-cupti-cu12==12.1.105 2024-08-20T22:34:44.1830101Z nvidia-cuda-nvrtc-cu12==12.1.105 2024-08-20T22:34:44.1830557Z nvidia-cuda-runtime-cu12==12.1.105 2024-08-20T22:34:44.1831003Z nvidia-cudnn-cu12==9.1.0.70 2024-08-20T22:34:44.1831404Z nvidia-cufft-cu12==11.0.2.54 2024-08-20T22:34:44.1831823Z nvidia-curand-cu12==10.3.2.106 2024-08-20T22:34:44.1832252Z nvidia-cusolver-cu12==11.4.5.107 2024-08-20T22:34:44.1832684Z nvidia-cusparse-cu12==12.1.0.106 2024-08-20T22:34:44.1833116Z nvidia-nccl-cu12==2.20.5 2024-08-20T22:34:44.1833520Z nvidia-nvjitlink-cu12==12.6.20 2024-08-20T22:34:44.1833941Z nvidia-nvtx-cu12==12.1.105 2024-08-20T22:34:44.1834281Z oauthlib==3.2.2 2024-08-20T22:34:44.1834618Z omegaconf==2.3.0 2024-08-20T22:34:44.1834925Z onnx==1.16.1 2024-08-20T22:34:44.1835216Z onnxscript==0.1.0.dev20240817 2024-08-20T22:34:44.1835578Z opacus==1.5.2 2024-08-20T22:34:44.1835923Z opencv-python==4.8.0.74 2024-08-20T22:34:44.1836297Z opt-einsum==3.3.0 2024-08-20T22:34:44.1836593Z optree==0.12.1 2024-08-20T22:34:44.1836892Z packaging==24.1 2024-08-20T22:34:44.1837188Z pandas==2.0.3 2024-08-20T22:34:44.1837475Z patch==1.16 2024-08-20T22:34:44.1837758Z pathspec==0.12.1 2024-08-20T22:34:44.1838068Z phonemizer==3.3.0 2024-08-20T22:34:44.1838370Z pillow==10.3.0 2024-08-20T22:34:44.1838662Z platformdirs==4.2.2 2024-08-20T22:34:44.1838977Z pluggy==1.5.0 2024-08-20T22:34:44.1839263Z pooch==1.8.2 2024-08-20T22:34:44.1839555Z portalocker==2.10.1 2024-08-20T22:34:44.1839865Z preshed==3.0.9 2024-08-20T22:34:44.1840162Z prettytable==3.11.0 2024-08-20T22:34:44.1840478Z protobuf==3.20.2 2024-08-20T22:34:44.1840769Z psutil==6.0.0 2024-08-20T22:34:44.1841100Z py-cpuinfo==9.0.0 2024-08-20T22:34:44.1841402Z pyaml==24.7.0 2024-08-20T22:34:44.1841693Z pyarrow==17.0.0 2024-08-20T22:34:44.1841982Z pyasn1==0.6.0 2024-08-20T22:34:44.1842275Z pyasn1_modules==0.4.0 2024-08-20T22:34:44.1842610Z pyclipper==1.3.0.post5 2024-08-20T22:34:44.1842935Z pycocotools==2.0.8 2024-08-20T22:34:44.1843362Z pycparser==2.22 2024-08-20T22:34:44.1843674Z pycryptodomex==3.20.0 2024-08-20T22:34:44.1844002Z pydantic==2.8.2 2024-08-20T22:34:44.1844299Z pydantic_core==2.20.1 2024-08-20T22:34:44.1844624Z pydot==3.0.1 2024-08-20T22:34:44.1844914Z pygame==2.6.0 2024-08-20T22:34:44.1845198Z PyGithub==2.3.0 2024-08-20T22:34:44.1845498Z Pygments==2.15.0 2024-08-20T22:34:44.1845798Z PyJWT==2.9.0 2024-08-20T22:34:44.1846092Z pylatexenc==2.10 2024-08-20T22:34:44.1846388Z pylint==3.2.6 2024-08-20T22:34:44.1846677Z PyNaCl==1.5.0 2024-08-20T22:34:44.1847441Z pynvml==11.5.3 2024-08-20T22:34:44.1847759Z pyparsing==3.1.2 2024-08-20T22:34:44.1848067Z pypdfium2==4.30.0 2024-08-20T22:34:44.1848375Z pysbd==0.3.4 2024-08-20T22:34:44.1848663Z PySocks==1.7.1 2024-08-20T22:34:44.1848949Z pytest==7.3.2 2024-08-20T22:34:44.1849313Z pytest-benchmark==4.0.0 2024-08-20T22:34:44.1849695Z pytest-cpp==2.3.0 2024-08-20T22:34:44.1850050Z pytest-flakefinder==1.1.0 2024-08-20T22:34:44.1850463Z pytest-rerunfailures==14.0 2024-08-20T22:34:44.1850865Z pytest-xdist==3.3.1 2024-08-20T22:34:44.1851238Z python-dateutil==2.9.0.post0 2024-08-20T22:34:44.1852032Z python-doctr @ git+https://github.com/xuzhao9/doctr.git@7f8a3e61bd6480c4e7b7cde47af0aa2153309c56 2024-08-20T22:34:44.1853492Z pytorch-labs-segment-anything-fast @ git+https://github.com/pytorch-labs/segment-anything-fast.git@de861af9badd03e8b40f5e063d70754a3dc6b4f4 2024-08-20T22:34:44.1854480Z pytz==2024.1 2024-08-20T22:34:44.1854776Z PyWavelets==1.4.1 2024-08-20T22:34:44.1855155Z PyYAML @ file:///croot/pyyaml_1698096049011/work 2024-08-20T22:34:44.1855609Z rapidfuzz==3.9.6 2024-08-20T22:34:44.1855909Z rdflib==7.0.0 2024-08-20T22:34:44.1856194Z redis==5.0.8 2024-08-20T22:34:44.1856477Z referencing==0.35.1 2024-08-20T22:34:44.1856792Z regex==2024.7.24 2024-08-20T22:34:44.1857091Z requests==2.32.3 2024-08-20T22:34:44.1857439Z requests-oauthlib==2.0.0 2024-08-20T22:34:44.1857787Z resampy==0.4.3 2024-08-20T22:34:44.1858076Z rfc3986==1.5.0 2024-08-20T22:34:44.1858369Z rich==13.7.1 2024-08-20T22:34:44.1858643Z rockset==1.0.3 2024-08-20T22:34:44.1858971Z rpds-py==0.20.0 2024-08-20T22:34:44.1859264Z rsa==4.9 2024-08-20T22:34:44.1859530Z s3transfer==0.5.2 2024-08-20T22:34:44.1859842Z safetensors==0.4.4 2024-08-20T22:34:44.1860194Z scikit-image==0.21.0 2024-08-20T22:34:44.1860554Z scikit-learn==1.5.1 2024-08-20T22:34:44.1860861Z scipy==1.10.1 2024-08-20T22:34:44.1861149Z segments==2.2.1 2024-08-20T22:34:44.1861454Z sentencepiece==0.2.0 2024-08-20T22:34:44.1861807Z sentry-sdk==2.13.0 2024-08-20T22:34:44.1862211Z setproctitle==1.3.3 2024-08-20T22:34:44.1862527Z shapely==2.0.6 2024-08-20T22:34:44.1862825Z shellingham==1.5.4 2024-08-20T22:34:44.1863138Z simplejson==3.19.3 2024-08-20T22:34:44.1863539Z six @ file:///tmp/build/80754af9/six_1644875935023/work 2024-08-20T22:34:44.1864051Z smart-open==7.0.4 2024-08-20T22:34:44.1864350Z smmap==5.0.1 2024-08-20T22:34:44.1864654Z sortedcontainers==2.4.0 2024-08-20T22:34:44.1864993Z soundfile==0.12.1 2024-08-20T22:34:44.1865297Z soupsieve==2.6 2024-08-20T22:34:44.1865586Z soxr==0.4.0 2024-08-20T22:34:44.1865862Z spacy==3.7.6 2024-08-20T22:34:44.1866193Z spacy-legacy==3.0.12 2024-08-20T22:34:44.1866550Z spacy-loggers==1.0.5 2024-08-20T22:34:44.1866871Z srsly==2.4.8 2024-08-20T22:34:44.1867155Z stempeg==0.2.3 2024-08-20T22:34:44.1867451Z submitit==1.5.1 2024-08-20T22:34:44.1867737Z sympy==1.13.1 2024-08-20T22:34:44.1868025Z tabulate==0.9.0 2024-08-20T22:34:44.1868371Z tb-nightly==2.13.0a20230426 2024-08-20T22:34:44.1868726Z tensorboard==2.13.0 2024-08-20T22:34:44.1869099Z tensorboard-data-server==0.7.2 2024-08-20T22:34:44.1869485Z tensorboardX==2.6.2.2 2024-08-20T22:34:44.1869814Z termcolor==2.4.0 2024-08-20T22:34:44.1870106Z thinc==8.2.5 2024-08-20T22:34:44.1870403Z threadpoolctl==3.5.0 2024-08-20T22:34:44.1870726Z tifffile==2024.8.10 2024-08-20T22:34:44.1871543Z timm @ git+https://github.com/huggingface/pytorch-image-models.git@730b907b4d45a4713cbc425cbf224c46089fd514 2024-08-20T22:34:44.1872333Z tlparse==0.3.25 2024-08-20T22:34:44.1872634Z tokenizers==0.15.2 2024-08-20T22:34:44.1873092Z tomli==2.0.1 2024-08-20T22:34:44.1873371Z tomlkit==0.13.2 2024-08-20T22:34:44.1874660Z torch @ file:///var/lib/jenkins/workspace/dist/torch-2.5.0a0%2Bgit40ec5f6-cp310-cp310-linux_x86_64.whl#sha256=b70a70c0c1fc7cc3f43c6342d758f4ffd3a540a88b786d266604d87618be5127 2024-08-20T22:34:44.1876336Z torch_geometric @ git+https://github.com/pyg-team/pytorch_geometric.git@cabcd4097442ba60aa1efa11e1619dd9bb8fb527 2024-08-20T22:34:44.1877430Z torchao @ git+https://github.com/pytorch/ao.git@2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T22:34:44.1878565Z torchaudio @ git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:34:44.1879583Z torchdata @ git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:34:44.1880719Z torchmultimodal @ git+https://github.com/facebookresearch/multimodal.git@e4d288b45b89cee462a21ab264405f3f368adc21 2024-08-20T22:34:44.1881833Z torchtext @ git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:34:44.1882850Z torchvision @ git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:34:44.1883553Z tornado==6.4.1 2024-08-20T22:34:44.1883850Z tqdm==4.66.5 2024-08-20T22:34:44.1884142Z transformers==4.38.1 2024-08-20T22:34:44.1884463Z treetable==0.2.5 2024-08-20T22:34:44.1885280Z -e git+https://github.com/openai/triton@dedb7bdf339a3546896d4820366ca562c586bfa0#egg=triton&subdirectory=python 2024-08-20T22:34:44.1886084Z typeguard==4.3.0 2024-08-20T22:34:44.1886391Z typer==0.12.4 2024-08-20T22:34:44.1886699Z typing_extensions==4.12.2 2024-08-20T22:34:44.1887037Z tzdata==2024.1 2024-08-20T22:34:44.1887333Z Unidecode==1.3.8 2024-08-20T22:34:44.1887704Z unittest-xml-reporting==3.2.0 2024-08-20T22:34:44.1888076Z uritemplate==4.1.1 2024-08-20T22:34:44.1888388Z urllib3==1.26.19 2024-08-20T22:34:44.1888685Z visdom==0.2.4 2024-08-20T22:34:44.1888976Z wandb==0.17.7 2024-08-20T22:34:44.1889252Z wasabi==1.1.3 2024-08-20T22:34:44.1889544Z wcwidth==0.2.13 2024-08-20T22:34:44.1889841Z weasel==0.4.1 2024-08-20T22:34:44.1890177Z websocket-client==1.8.0 2024-08-20T22:34:44.1890518Z Werkzeug==3.0.3 2024-08-20T22:34:44.1890818Z wrapt==1.16.0 2024-08-20T22:34:44.1891107Z xdoctest==1.1.0 2024-08-20T22:34:44.1891395Z xxhash==3.5.0 2024-08-20T22:34:44.1891693Z xyzservices==2024.6.0 2024-08-20T22:34:44.1892020Z yacs==0.1.8 2024-08-20T22:34:44.1892288Z yarl==1.9.4 2024-08-20T22:34:44.1892604Z z3-solver==4.12.2.0 2024-08-20T22:34:44.1892918Z zipp==3.20.0 2024-08-20T22:34:44.2071111Z + popd 2024-08-20T22:34:44.2071413Z ~/workspace 2024-08-20T22:34:44.2071757Z + [[ dynamo_eager_torchbench != *cpu* ]] 2024-08-20T22:34:44.2072207Z + install_torchrec_and_fbgemm 2024-08-20T22:34:44.2072593Z + local torchrec_commit 2024-08-20T22:34:44.2076039Z ++ get_pinned_commit torchrec 2024-08-20T22:34:44.2076475Z ++ cat .github/ci_commit_pins/torchrec.txt 2024-08-20T22:34:44.2096206Z + torchrec_commit=6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T22:34:44.2096750Z + local fbgemm_commit 2024-08-20T22:34:44.2100016Z ++ get_pinned_commit fbgemm 2024-08-20T22:34:44.2100442Z ++ cat .github/ci_commit_pins/fbgemm.txt 2024-08-20T22:34:44.2119388Z + fbgemm_commit=de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:34:44.2120010Z + pip_uninstall torchrec-nightly 2024-08-20T22:34:44.2120485Z + pip uninstall -y torchrec-nightly 2024-08-20T22:34:44.5829030Z WARNING: Skipping torchrec-nightly as it is not installed. 2024-08-20T22:34:44.6175229Z + pip_uninstall fbgemm-gpu-nightly 2024-08-20T22:34:44.6175843Z + pip uninstall -y fbgemm-gpu-nightly 2024-08-20T22:34:44.9885828Z WARNING: Skipping fbgemm-gpu-nightly as it is not installed. 2024-08-20T22:34:45.0234143Z + pip_install setuptools-git-versioning scikit-build pyre-extensions 2024-08-20T22:34:45.0235124Z + pip install --progress-bar off setuptools-git-versioning scikit-build pyre-extensions 2024-08-20T22:34:45.5414788Z Collecting setuptools-git-versioning 2024-08-20T22:34:45.5570517Z Downloading setuptools_git_versioning-2.0.0-py3-none-any.whl.metadata (5.8 kB) 2024-08-20T22:34:45.6701845Z Collecting scikit-build 2024-08-20T22:34:45.6726382Z Downloading scikit_build-0.18.0-py3-none-any.whl.metadata (17 kB) 2024-08-20T22:34:45.7887061Z Collecting pyre-extensions 2024-08-20T22:34:45.7915342Z Downloading pyre_extensions-0.0.30-py3-none-any.whl.metadata (4.0 kB) 2024-08-20T22:34:45.8422488Z Requirement already satisfied: packaging in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from setuptools-git-versioning) (24.1) 2024-08-20T22:34:45.8425821Z Requirement already satisfied: setuptools in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from setuptools-git-versioning) (72.1.0) 2024-08-20T22:34:45.9002196Z Collecting toml>=0.10.2 (from setuptools-git-versioning) 2024-08-20T22:34:45.9029789Z Downloading toml-0.10.2-py2.py3-none-any.whl.metadata (7.1 kB) 2024-08-20T22:34:45.9555156Z Requirement already satisfied: distro in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (1.9.0) 2024-08-20T22:34:45.9560133Z Requirement already satisfied: tomli in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (2.0.1) 2024-08-20T22:34:45.9563883Z Requirement already satisfied: wheel>=0.32.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (0.43.0) 2024-08-20T22:34:46.0171309Z Collecting typing-inspect (from pyre-extensions) 2024-08-20T22:34:46.0202634Z Downloading typing_inspect-0.9.0-py3-none-any.whl.metadata (1.5 kB) 2024-08-20T22:34:46.0697445Z Requirement already satisfied: typing-extensions in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pyre-extensions) (4.12.2) 2024-08-20T22:34:46.0777571Z Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from typing-inspect->pyre-extensions) (1.0.0) 2024-08-20T22:34:46.0815343Z Downloading setuptools_git_versioning-2.0.0-py3-none-any.whl (10 kB) 2024-08-20T22:34:46.1331338Z Downloading scikit_build-0.18.0-py3-none-any.whl (85 kB) 2024-08-20T22:34:46.1920908Z Downloading pyre_extensions-0.0.30-py3-none-any.whl (12 kB) 2024-08-20T22:34:46.2506828Z Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-08-20T22:34:46.3069124Z Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB) 2024-08-20T22:34:47.1976205Z Installing collected packages: typing-inspect, toml, scikit-build, setuptools-git-versioning, pyre-extensions 2024-08-20T22:34:47.5772063Z Successfully installed pyre-extensions-0.0.30 scikit-build-0.18.0 setuptools-git-versioning-2.0.0 toml-0.10.2 typing-inspect-0.9.0 2024-08-20T22:34:47.6870529Z + CUDA_PATH=/usr/local/cuda-12.1 2024-08-20T22:34:47.6871771Z + pip_install --no-use-pep517 --user 'git+https://github.com/pytorch/FBGEMM.git@de731af65b4f04696e85c729e3282450b51b95fd#egg=fbgemm-gpu&subdirectory=fbgemm_gpu' 2024-08-20T22:34:47.6873635Z + pip install --progress-bar off --no-use-pep517 --user 'git+https://github.com/pytorch/FBGEMM.git@de731af65b4f04696e85c729e3282450b51b95fd#egg=fbgemm-gpu&subdirectory=fbgemm_gpu' 2024-08-20T22:34:48.0574007Z Collecting fbgemm-gpu 2024-08-20T22:34:48.0575657Z Cloning https://github.com/pytorch/FBGEMM.git (to revision de731af65b4f04696e85c729e3282450b51b95fd) to /tmp/pip-install-knr1kqav/fbgemm-gpu_bab02720cb9c4e54a75d2326a6643b14 2024-08-20T22:34:48.0606231Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/FBGEMM.git /tmp/pip-install-knr1kqav/fbgemm-gpu_bab02720cb9c4e54a75d2326a6643b14 2024-08-20T22:34:49.1587080Z Running command git rev-parse -q --verify 'sha^de731af65b4f04696e85c729e3282450b51b95fd' 2024-08-20T22:34:49.1616070Z Running command git fetch -q https://github.com/pytorch/FBGEMM.git de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:34:49.6534424Z Running command git checkout -q de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:34:50.1689067Z Resolved https://github.com/pytorch/FBGEMM.git to commit de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:34:50.1690240Z Running command git submodule update --init --recursive -q 2024-08-20T22:34:59.5783295Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:34:59.5802591Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbgemm-gpu) (1.21.2) 2024-08-20T22:34:59.5810382Z Building wheels for collected packages: fbgemm-gpu 2024-08-20T23:19:33.4715476Z Building wheel for fbgemm-gpu (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-08-20T23:19:33.6545905Z [?25h Created wheel for fbgemm-gpu: filename=fbgemm_gpu-0.4.1rc0.post421-cp310-cp310-linux_x86_64.whl size=252618862 sha256=ea2538d0300c3236cd68d0ba04831feef2107549c19610326d6a2055412bf750 2024-08-20T23:19:33.6547781Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/d8/d9/f3/f5c260aba9c9f0a533444c12621a0c2ceb8fbcffce2b5beb39 2024-08-20T23:19:33.6568781Z Successfully built fbgemm-gpu 2024-08-20T23:19:34.5078173Z Installing collected packages: fbgemm-gpu 2024-08-20T23:19:37.8405434Z Successfully installed fbgemm-gpu-0.4.1rc0.post421 2024-08-20T23:19:38.7564653Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:38.7566517Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:39.1273181Z Collecting git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:39.1275544Z Cloning https://github.com/pytorch/torchrec.git (to revision 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2) to /tmp/pip-req-build-1m4_u2kk 2024-08-20T23:19:39.1303834Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/torchrec.git /tmp/pip-req-build-1m4_u2kk 2024-08-20T23:19:39.8691198Z Running command git rev-parse -q --verify 'sha^6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2' 2024-08-20T23:19:39.8720060Z Running command git fetch -q https://github.com/pytorch/torchrec.git 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:40.1794726Z Running command git checkout -q 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:40.5281280Z Resolved https://github.com/pytorch/torchrec.git to commit 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:40.8002154Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T23:19:40.8038270Z [?25hRequirement already satisfied: iopath in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (0.1.9) 2024-08-20T23:19:40.8041082Z Requirement already satisfied: pandas in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (2.0.3) 2024-08-20T23:19:40.8045422Z Requirement already satisfied: tabulate in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (0.9.0) 2024-08-20T23:19:40.8530050Z Collecting torchmetrics (from torchrec==0.3.2) 2024-08-20T23:19:40.8698101Z Downloading torchmetrics-1.4.1-py3-none-any.whl.metadata (20 kB) 2024-08-20T23:19:40.9091867Z Requirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (4.66.5) 2024-08-20T23:19:40.9093711Z Requirement already satisfied: fbgemm-gpu in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchrec==0.3.2) (0.4.1rc0.post421) 2024-08-20T23:19:40.9103569Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbgemm-gpu->torchrec==0.3.2) (1.21.2) 2024-08-20T23:19:40.9113025Z Requirement already satisfied: portalocker in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from iopath->torchrec==0.3.2) (2.10.1) 2024-08-20T23:19:40.9380682Z Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2.9.0.post0) 2024-08-20T23:19:40.9384280Z Requirement already satisfied: pytz>=2020.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2024.1) 2024-08-20T23:19:40.9387937Z Requirement already satisfied: tzdata>=2022.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2024.1) 2024-08-20T23:19:40.9409717Z Requirement already satisfied: packaging>17.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchmetrics->torchrec==0.3.2) (24.1) 2024-08-20T23:19:40.9413651Z Requirement already satisfied: torch>=1.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchmetrics->torchrec==0.3.2) (2.5.0a0+git40ec5f6) 2024-08-20T23:19:40.9561983Z Collecting lightning-utilities>=0.8.0 (from torchmetrics->torchrec==0.3.2) 2024-08-20T23:19:40.9587869Z Downloading lightning_utilities-0.11.6-py3-none-any.whl.metadata (5.2 kB) 2024-08-20T23:19:40.9678118Z Requirement already satisfied: setuptools in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from lightning-utilities>=0.8.0->torchmetrics->torchrec==0.3.2) (72.1.0) 2024-08-20T23:19:40.9681834Z Requirement already satisfied: typing-extensions in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from lightning-utilities>=0.8.0->torchmetrics->torchrec==0.3.2) (4.12.2) 2024-08-20T23:19:40.9702851Z Requirement already satisfied: six>=1.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->torchrec==0.3.2) (1.16.0) 2024-08-20T23:19:40.9737566Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (3.13.1) 2024-08-20T23:19:40.9742397Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2.8.8) 2024-08-20T23:19:40.9745884Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (3.1.4) 2024-08-20T23:19:40.9747949Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2024.6.1) 2024-08-20T23:19:40.9753567Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (1.13.1) 2024-08-20T23:19:40.9769005Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch>=1.10.0->torchmetrics->torchrec==0.3.2) (1.3.0) 2024-08-20T23:19:41.0187301Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2.1.5) 2024-08-20T23:19:41.0322276Z Downloading torchmetrics-1.4.1-py3-none-any.whl (866 kB) 2024-08-20T23:19:41.0557491Z Downloading lightning_utilities-0.11.6-py3-none-any.whl (26 kB) 2024-08-20T23:19:41.0667955Z Building wheels for collected packages: torchrec 2024-08-20T23:19:41.5894861Z Building wheel for torchrec (setup.py) ... [?25l- \ | done 2024-08-20T23:19:41.5903614Z [?25h Created wheel for torchrec: filename=torchrec-0.3.2-py3-none-any.whl size=374488 sha256=73b56428805d07a95f25fcc5e30164a22cff8c1bf4fef492ce907787e06b8512 2024-08-20T23:19:41.5905403Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/8e/a1/47/39ede01672ba82c08fb521bfc057cc4347e4b0e951738c8ca8 2024-08-20T23:19:41.5927535Z Successfully built torchrec 2024-08-20T23:19:42.3836542Z Installing collected packages: lightning-utilities, torchmetrics, torchrec 2024-08-20T23:19:42.9923884Z Successfully installed lightning-utilities-0.11.6 torchmetrics-1.4.1 torchrec-0.3.2 2024-08-20T23:19:43.1339576Z ++ pwd 2024-08-20T23:19:43.1342539Z + PYTHONPATH=/var/lib/jenkins/workspace/torchbench 2024-08-20T23:19:43.1343070Z + test_dynamo_benchmark torchbench 1 2024-08-20T23:19:43.1347913Z ++ pwd 2024-08-20T23:19:43.1352080Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:19:43.1352646Z + local suite=torchbench 2024-08-20T23:19:43.1352989Z + shift 2024-08-20T23:19:43.1355700Z + local shard_id=1 2024-08-20T23:19:43.1356077Z + shift 2024-08-20T23:19:43.1356521Z + [[ dynamo_eager_torchbench == *perf_compare* ]] 2024-08-20T23:19:43.1357164Z + [[ dynamo_eager_torchbench == *perf* ]] 2024-08-20T23:19:43.1357646Z + [[ dynamo_eager_torchbench == *cpu* ]] 2024-08-20T23:19:43.1358140Z + [[ dynamo_eager_torchbench == *aot_inductor* ]] 2024-08-20T23:19:43.1359071Z + test_single_dynamo_benchmark inference torchbench 1 --inference --bfloat16 2024-08-20T23:19:43.1362112Z ++ pwd 2024-08-20T23:19:43.1362679Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:19:43.1363380Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:19:43.1411580Z + local name=inference 2024-08-20T23:19:43.1412084Z + shift 2024-08-20T23:19:43.1412447Z + local suite=torchbench 2024-08-20T23:19:43.1412852Z + shift 2024-08-20T23:19:43.1413212Z + local shard_id=1 2024-08-20T23:19:43.1413523Z + shift 2024-08-20T23:19:43.1413801Z + partition_flags=() 2024-08-20T23:19:43.1414135Z + local partition_flags 2024-08-20T23:19:43.1414538Z + [[ -n 2 ]] 2024-08-20T23:19:43.1414855Z + [[ -n 1 ]] 2024-08-20T23:19:43.1415902Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T23:19:43.1416787Z + [[ dynamo_eager_torchbench == *perf_compare* ]] 2024-08-20T23:19:43.1417342Z + [[ dynamo_eager_torchbench == *perf* ]] 2024-08-20T23:19:43.1417950Z + [[ dynamo_eager_torchbench == *aot_inductor* ]] 2024-08-20T23:19:43.1418620Z + [[ dynamo_eager_torchbench == *_avx2* ]] 2024-08-20T23:19:43.1419185Z + [[ dynamo_eager_torchbench == *_avx512* ]] 2024-08-20T23:19:43.1420894Z + python benchmarks/dynamo/torchbench.py --ci --accuracy --timing --explain --backend eager --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 1 --output /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv 2024-08-20T23:19:49.4102064Z 2024-08-20T23:19:50.2072923Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:19:50.2073541Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:19:50.2107253Z cuda eval llama 2024-08-20T23:19:53.5994652Z pass 2024-08-20T23:19:53.6166406Z TIMING: backend_compile:0.0004 entire_frame_compile:2.8113 total_wall_time:2.8113 2024-08-20T23:19:53.6167370Z STATS: call_* op count: 1058 | FakeTensorMode.__torch_dispatch__:2673 2024-08-20T23:19:53.6168155Z Dynamo produced 1 graphs covering 1058 ops with 0 graph breaks (0 unique) 2024-08-20T23:19:57.2432389Z 2024-08-20T23:19:57.5487767Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:19:57.5488381Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:19:57.5488905Z cuda eval llama_v2_7b_16h 2024-08-20T23:19:57.5510393Z Traceback (most recent call last): 2024-08-20T23:19:57.5511238Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:19:57.5512013Z ) = runner.load_model( 2024-08-20T23:19:57.5512685Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 309, in load_model 2024-08-20T23:19:57.5513397Z benchmark = benchmark_cls( 2024-08-20T23:19:57.5514120Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 24, in __call__ 2024-08-20T23:19:57.5515114Z obj = type.__call__(cls, *args, **kwargs) 2024-08-20T23:19:57.5516033Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama_v2_7b_16h/__init__.py", line 12, in __init__ 2024-08-20T23:19:57.5517222Z super().__init__(name="llama_v2_7b_16h", test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:19:57.5518535Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 81, in __init__ 2024-08-20T23:19:57.5519736Z super().__init__(test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:19:57.5520722Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 92, in __init__ 2024-08-20T23:19:57.5521483Z self._skip_by_device_name() 2024-08-20T23:19:57.5522291Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 165, in _skip_by_device_name 2024-08-20T23:19:57.5523556Z raise NotImplementedError(f"The current device {current_device_name} is skipped by its `{self.name}/metadata.yaml`.") 2024-08-20T23:19:57.5524714Z NotImplementedError: The current device NVIDIA A10G is skipped by its `llama_v2_7b_16h/metadata.yaml`. 2024-08-20T23:19:57.5525327Z 2024-08-20T23:19:57.5525454Z model_fail_to_load 2024-08-20T23:20:01.0142251Z 2024-08-20T23:20:01.3187010Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:20:01.3188023Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:20:01.3188918Z cuda eval llava 2024-08-20T23:20:01.3192925Z Traceback (most recent call last): 2024-08-20T23:20:01.3193838Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:20:01.3194717Z ) = runner.load_model( 2024-08-20T23:20:01.3195411Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 309, in load_model 2024-08-20T23:20:01.3196137Z benchmark = benchmark_cls( 2024-08-20T23:20:01.3196881Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 24, in __call__ 2024-08-20T23:20:01.3197673Z obj = type.__call__(cls, *args, **kwargs) 2024-08-20T23:20:01.3198535Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llava/__init__.py", line 11, in __init__ 2024-08-20T23:20:01.3199746Z super().__init__(name="llava", test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:20:01.3201059Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 81, in __init__ 2024-08-20T23:20:01.3202455Z super().__init__(test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:20:01.3203454Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 92, in __init__ 2024-08-20T23:20:01.3204210Z self._skip_by_device_name() 2024-08-20T23:20:01.3205031Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 165, in _skip_by_device_name 2024-08-20T23:20:01.3206272Z raise NotImplementedError(f"The current device {current_device_name} is skipped by its `{self.name}/metadata.yaml`.") 2024-08-20T23:20:01.3207365Z NotImplementedError: The current device NVIDIA A10G is skipped by its `llava/metadata.yaml`. 2024-08-20T23:20:01.3207934Z 2024-08-20T23:20:01.3208056Z model_fail_to_load 2024-08-20T23:20:04.7735142Z 2024-08-20T23:20:05.4940943Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:20:05.4942084Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:20:05.4946525Z cuda eval maml 2024-08-20T23:20:05.4949859Z pass_due_to_skip 2024-08-20T23:20:05.4958318Z TIMING: 2024-08-20T23:20:05.4958724Z STATS: call_* op count: 0 2024-08-20T23:20:05.4959285Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:20:09.0704630Z 2024-08-20T23:20:09.6091329Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:20:09.6091879Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:20:09.6097646Z cuda eval maml_omniglot 2024-08-20T23:20:10.2136948Z pass 2024-08-20T23:20:10.2137640Z TIMING: backend_compile:0.0001 entire_frame_compile:0.39657 total_wall_time:0.39657 2024-08-20T23:20:10.2138453Z STATS: call_* op count: 28 | FakeTensorMode.__torch_dispatch__:242 2024-08-20T23:20:10.2139201Z Dynamo produced 1 graphs covering 28 ops with 0 graph breaks (0 unique) 2024-08-20T23:20:13.8435799Z 2024-08-20T23:20:14.0684877Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/mnasnet1.0_top1_73.512-f206786ef8.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/mnasnet1.0_top1_73.512-f206786ef8.pth 2024-08-20T23:20:14.0866257Z 2024-08-20T23:20:14.0866576Z 2024-08-20T23:20:14.1351567Z 0% 0.00/16.9M [00:00 will be ignored 2024-08-20T23:21:03.8942061Z [rank0]:W0820 23:21:03.893000 32563 torch/_dynamo/backends/distributed.py:89] [2/0_1] Some buckets were extended beyond their requested parameter capacities in order to ensure each subgraph has an output node, required for fx graph partitioning. This can be the case when a subgraph would have only contained nodes performing inplace mutation, and returning no logical outputs. This should not be a problem, unless it results in too few graph partitions for optimal DDP performance. 2024-08-20T23:21:03.9192670Z [rank0]:W0820 23:21:03.918000 32563 torch/_dynamo/backends/distributed.py:106] [2/0_1] DDPOptimizer extended these buckets to ensure per-subgraph output nodes: 2024-08-20T23:21:03.9196096Z [rank0]:W0820 23:21:03.918000 32563 torch/_dynamo/backends/distributed.py:106] [2/0_1] ┌─────────┬─────────────┬────────────────────────┐ 2024-08-20T23:21:03.9197663Z [rank0]:W0820 23:21:03.918000 32563 torch/_dynamo/backends/distributed.py:106] [2/0_1] │ Index │ Extra Ops │ Extra Param Size (b) │ 2024-08-20T23:21:03.9199147Z [rank0]:W0820 23:21:03.918000 32563 torch/_dynamo/backends/distributed.py:106] [2/0_1] ├─────────┼─────────────┼────────────────────────┤ 2024-08-20T23:21:03.9200564Z [rank0]:W0820 23:21:03.918000 32563 torch/_dynamo/backends/distributed.py:106] [2/0_1] │ 0 │ 470 │ 44910720 │ 2024-08-20T23:21:03.9202036Z [rank0]:W0820 23:21:03.918000 32563 torch/_dynamo/backends/distributed.py:106] [2/0_1] └─────────┴─────────────┴────────────────────────┘ 2024-08-20T23:21:12.2406849Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] Graph break from `Tensor.item()`, consider setting: 2024-08-20T23:21:12.2408536Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] torch._dynamo.config.capture_scalar_outputs = True 2024-08-20T23:21:12.2409655Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] or: 2024-08-20T23:21:12.2410680Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-08-20T23:21:12.2412026Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] to include these operations in the captured graph. 2024-08-20T23:21:12.2413086Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] 2024-08-20T23:21:12.2414057Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] Graph break: from user code at: 2024-08-20T23:21:12.2415774Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco/moco/builder.py", line 158, in torch_dynamo_resume_in_forward_at_133 2024-08-20T23:21:12.2417486Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] self._dequeue_and_enqueue(k) 2024-08-20T23:21:12.2419355Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context 2024-08-20T23:21:12.2420930Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] return func(*args, **kwargs) 2024-08-20T23:21:12.2422674Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco/moco/builder.py", line 59, in _dequeue_and_enqueue 2024-08-20T23:21:12.2424626Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] ptr = int(self.queue_ptr) 2024-08-20T23:21:12.2425560Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] 2024-08-20T23:21:12.2426338Z [rank0]:W0820 23:21:12.239000 32563 torch/_dynamo/variables/tensor.py:776] [5/0] 2024-08-20T23:21:16.4141657Z pass 2024-08-20T23:21:16.4158357Z TIMING: entire_frame_compile:23.51576 backend_compile:3.438 total_wall_time:23.51576 2024-08-20T23:21:16.4159348Z STATS: call_* op count: 879 | FakeTensorMode.__torch_dispatch__:10411 | FakeTensor.__torch_dispatch__:4 2024-08-20T23:21:16.4160281Z Dynamo produced 7 graphs covering 879 ops with 5 graph breaks (3 unique) 2024-08-20T23:21:16.8351287Z [rank0]:[W820 23:21:16.799675621 ProcessGroupNCCL.cpp:1223] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator()) 2024-08-20T23:21:20.9314011Z 2024-08-20T23:21:21.2367750Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:21.2368331Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:21.2368857Z cuda eval moondream 2024-08-20T23:21:21.2375130Z Traceback (most recent call last): 2024-08-20T23:21:21.2375974Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:21:21.2376662Z ) = runner.load_model( 2024-08-20T23:21:21.2377354Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 309, in load_model 2024-08-20T23:21:21.2378321Z benchmark = benchmark_cls( 2024-08-20T23:21:21.2379131Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 24, in __call__ 2024-08-20T23:21:21.2380234Z obj = type.__call__(cls, *args, **kwargs) 2024-08-20T23:21:21.2381259Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moondream/__init__.py", line 11, in __init__ 2024-08-20T23:21:21.2382701Z super().__init__(name="moondream", test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:21:21.2384042Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 81, in __init__ 2024-08-20T23:21:21.2385266Z super().__init__(test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:21:21.2386297Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 92, in __init__ 2024-08-20T23:21:21.2387095Z self._skip_by_device_name() 2024-08-20T23:21:21.2387932Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 165, in _skip_by_device_name 2024-08-20T23:21:21.2389208Z raise NotImplementedError(f"The current device {current_device_name} is skipped by its `{self.name}/metadata.yaml`.") 2024-08-20T23:21:21.2390368Z NotImplementedError: The current device NVIDIA A10G is skipped by its `moondream/metadata.yaml`. 2024-08-20T23:21:21.2390977Z 2024-08-20T23:21:21.2391108Z model_fail_to_load 2024-08-20T23:21:24.7034752Z 2024-08-20T23:21:26.7251217Z loading model: 0it [00:00, ?it/s]number of parameters: 123.69M 2024-08-20T23:21:26.9720525Z num decayed parameter tensors: 50, with 124,354,560 parameters 2024-08-20T23:21:26.9721672Z num non-decayed parameter tensors: 98, with 121,344 parameters 2024-08-20T23:21:26.9722635Z using fused AdamW: True 2024-08-20T23:21:27.3176040Z 2024-08-20T23:21:27.3176306Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:21:27.3215074Z cuda eval nanogpt 2024-08-20T23:21:30.6254225Z pass 2024-08-20T23:21:30.6261186Z TIMING: backend_compile:0.00043 entire_frame_compile:2.72065 total_wall_time:2.72065 2024-08-20T23:21:30.6262315Z STATS: call_* op count: 786 | FakeTensorMode.__torch_dispatch__:2188 | FakeTensor.__torch_dispatch__:72 2024-08-20T23:21:30.6263321Z Dynamo produced 1 graphs covering 786 ops with 0 graph breaks (0 unique) 2024-08-20T23:21:34.2772093Z 2024-08-20T23:21:36.6785820Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:36.6786496Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:21:36.6789400Z cuda eval nvidia_deeprecommender 2024-08-20T23:21:37.3207923Z pass 2024-08-20T23:21:37.3242654Z TIMING: backend_compile:9e-05 entire_frame_compile:0.33285 total_wall_time:0.33285 2024-08-20T23:21:37.3243472Z STATS: call_* op count: 26 | FakeTensorMode.__torch_dispatch__:138 2024-08-20T23:21:37.3244232Z Dynamo produced 1 graphs covering 26 ops with 0 graph breaks (0 unique) 2024-08-20T23:21:40.8326950Z 2024-08-20T23:21:41.7645772Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:41.7646696Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:41.7661831Z cuda eval opacus_cifar10 2024-08-20T23:21:44.2720877Z pass 2024-08-20T23:21:44.2721575Z TIMING: backend_compile:0.00018 entire_frame_compile:2.15283 total_wall_time:2.15283 2024-08-20T23:21:44.2722391Z STATS: call_* op count: 138 | FakeTensorMode.__torch_dispatch__:268 2024-08-20T23:21:44.2723146Z Dynamo produced 1 graphs covering 138 ops with 0 graph breaks (0 unique) 2024-08-20T23:21:47.9163415Z 2024-08-20T23:21:48.5653791Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:48.5654691Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:48.5715227Z cuda eval phlippe_densenet 2024-08-20T23:21:54.2073021Z pass 2024-08-20T23:21:54.2075349Z TIMING: backend_compile:0.00057 entire_frame_compile:5.25421 total_wall_time:5.25421 2024-08-20T23:21:54.2076183Z STATS: call_* op count: 372 | FakeTensorMode.__torch_dispatch__:3576 2024-08-20T23:21:54.2076976Z Dynamo produced 1 graphs covering 372 ops with 0 graph breaks (0 unique) 2024-08-20T23:21:57.8713328Z 2024-08-20T23:21:58.4183704Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:58.4184192Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:21:58.4209608Z cuda eval phlippe_resnet 2024-08-20T23:22:00.4924909Z pass 2024-08-20T23:22:00.4926531Z TIMING: backend_compile:0.00025 entire_frame_compile:1.7943 total_wall_time:1.7943 2024-08-20T23:22:00.4928145Z STATS: call_* op count: 142 | FakeTensorMode.__torch_dispatch__:1300 2024-08-20T23:22:00.4929720Z Dynamo produced 1 graphs covering 142 ops with 0 graph breaks (0 unique) 2024-08-20T23:22:04.0959321Z 2024-08-20T23:22:04.4541840Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:04.4542317Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:04.4542801Z cuda eval pyhpc_equation_of_state 2024-08-20T23:22:05.2287193Z pass 2024-08-20T23:22:05.2287821Z TIMING: backend_compile:0.00018 entire_frame_compile:0.57597 total_wall_time:0.57597 2024-08-20T23:22:05.2289422Z STATS: call_* op count: 732 | FakeTensorMode.__torch_dispatch__:744 2024-08-20T23:22:05.2290257Z Dynamo produced 1 graphs covering 732 ops with 0 graph breaks (0 unique) 2024-08-20T23:22:08.7635968Z 2024-08-20T23:22:09.0784395Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:09.0784891Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:09.0788729Z cuda eval pyhpc_isoneutral_mixing 2024-08-20T23:22:11.3547851Z pass 2024-08-20T23:22:11.3548611Z TIMING: backend_compile:0.00033 entire_frame_compile:2.05813 total_wall_time:2.05813 2024-08-20T23:22:11.3549550Z STATS: call_* op count: 1488 | FakeTensorMode.__torch_dispatch__:4719 2024-08-20T23:22:11.3550321Z Dynamo produced 1 graphs covering 1488 ops with 0 graph breaks (0 unique) 2024-08-20T23:22:14.9467029Z 2024-08-20T23:22:15.7075842Z loading model: 0it [00:00, ?it/s]WARNING:common:Model pyhpc_turbulent_kinetic_energy does not support bfloat16, running with amp instead 2024-08-20T23:22:15.8249352Z 2024-08-20T23:22:15.8250177Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:15.8250994Z WARNING:common:Model pyhpc_turbulent_kinetic_energy does not support bfloat16, running with amp instead 2024-08-20T23:22:15.8251792Z cuda eval pyhpc_turbulent_kinetic_energy 2024-08-20T23:22:15.8701732Z WARNING:common:Model pyhpc_turbulent_kinetic_energy does not support bfloat16, running with amp instead 2024-08-20T23:22:18.4026021Z pass 2024-08-20T23:22:18.4129692Z TIMING: backend_compile:0.00041 entire_frame_compile:2.26387 total_wall_time:2.26387 2024-08-20T23:22:18.4132485Z STATS: call_* op count: 1992 | FakeTensorMode.__torch_dispatch__:5445 | FakeTensor.__torch_dispatch__:1 2024-08-20T23:22:18.4134340Z Dynamo produced 1 graphs covering 1992 ops with 0 graph breaks (0 unique) 2024-08-20T23:22:22.0012050Z 2024-08-20T23:22:22.7962902Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:22.7974650Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:22.7975258Z cuda eval pytorch_CycleGAN_and_pix2pix 2024-08-20T23:22:30.1835840Z pass 2024-08-20T23:22:30.1846660Z TIMING: backend_compile:0.00017 entire_frame_compile:1.70795 total_wall_time:1.70795 2024-08-20T23:22:30.1848105Z STATS: call_* op count: 182 | FakeTensorMode.__torch_dispatch__:1082 2024-08-20T23:22:30.1848877Z Dynamo produced 1 graphs covering 182 ops with 0 graph breaks (0 unique) 2024-08-20T23:22:33.8555132Z 2024-08-20T23:22:35.0020849Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:35.0021573Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:22:35.0040832Z cuda eval pytorch_stargan 2024-08-20T23:22:51.3935596Z pass 2024-08-20T23:22:51.3979333Z TIMING: backend_compile:0.00021 entire_frame_compile:1.68926 total_wall_time:1.68926 2024-08-20T23:22:51.3980312Z STATS: call_* op count: 112 | FakeTensorMode.__torch_dispatch__:1806 | FakeTensor.__torch_dispatch__:6 2024-08-20T23:22:51.3983302Z Dynamo produced 1 graphs covering 112 ops with 0 graph breaks (0 unique) 2024-08-20T23:22:55.0407479Z 2024-08-20T23:22:55.7434398Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:55.7435028Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:22:55.7455777Z cuda eval pytorch_unet 2024-08-20T23:23:05.2466997Z pass 2024-08-20T23:23:05.2554004Z TIMING: backend_compile:0.00074 entire_frame_compile:2.04587 total_wall_time:2.04587 2024-08-20T23:23:05.2555004Z STATS: call_* op count: 142 | FakeTensorMode.__torch_dispatch__:1604 2024-08-20T23:23:05.2555757Z Dynamo produced 1 graphs covering 142 ops with 0 graph breaks (0 unique) 2024-08-20T23:23:08.8213232Z 2024-08-20T23:23:09.6300157Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/resnet152-394f9c45.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/resnet152-394f9c45.pth 2024-08-20T23:23:09.6505217Z 2024-08-20T23:23:09.6505305Z 2024-08-20T23:23:09.7511445Z 0% 0.00/230M [00:00 2024-08-20T23:35:06.6173861Z W0820 23:35:06.612000 37257 torch/_dynamo/variables/tensor.py:776] [13/0] scales = [_infer_scale(feat, original_input_shape) for feat in features] 2024-08-20T23:35:06.6176934Z W0820 23:35:06.612000 37257 torch/_dynamo/variables/tensor.py:776] [13/0] File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/poolers.py", line 104, in _infer_scale 2024-08-20T23:35:06.6179715Z W0820 23:35:06.612000 37257 torch/_dynamo/variables/tensor.py:776] [13/0] scale = 2 ** float(torch.tensor(approx_scale).log2().round()) 2024-08-20T23:35:06.6181606Z W0820 23:35:06.612000 37257 torch/_dynamo/variables/tensor.py:776] [13/0] 2024-08-20T23:35:06.6182858Z W0820 23:35:06.612000 37257 torch/_dynamo/variables/tensor.py:776] [13/0] 2024-08-20T23:35:28.0926507Z pass 2024-08-20T23:35:28.1014252Z TIMING: entire_frame_compile:35.98543 backend_compile:0.0023 total_wall_time:35.98543 2024-08-20T23:35:28.1016294Z STATS: call_* op count: 1358 | FakeTensorMode.__torch_dispatch__:14852 | FakeTensor.__torch_dispatch__:37 | attempt fast:68 | fast is_contiguous:48 | slow both tensors nontrivially broadcast:20 2024-08-20T23:35:28.1018286Z Dynamo produced 25 graphs covering 1358 ops with 16 graph breaks (2 unique) 2024-08-20T23:35:32.1051946Z 2024-08-20T23:35:34.3200940Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:35:34.3201406Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:35:34.3302134Z cuda eval yolov3 2024-08-20T23:35:48.4516321Z pass 2024-08-20T23:35:48.4630284Z TIMING: backend_compile:0.00081 entire_frame_compile:7.4735 total_wall_time:7.4735 2024-08-20T23:35:48.4631861Z STATS: call_* op count: 649 | FakeTensorMode.__torch_dispatch__:5179 | FakeTensor.__torch_dispatch__:32 2024-08-20T23:35:48.4632757Z Dynamo produced 1 graphs covering 649 ops with 0 graph breaks (0 unique) 2024-08-20T23:35:49.5122265Z accuracy pass_rate=82.61% 2024-08-20T23:35:49.5125933Z calls_captured gmean=0.00x mean=662.391x 2024-08-20T23:35:49.5129430Z unique_graphs gmean=0.00x mean=1.674x 2024-08-20T23:35:49.5133088Z graph_breaks gmean=0.00x mean=0.717x 2024-08-20T23:35:49.5135938Z unique_graph_breaks gmean=0.00x mean=0.174x 2024-08-20T23:35:49.5139157Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T23:35:49.5142575Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T23:35:49.5145903Z cudagraph_skips gmean=0.00x mean=0.000x 2024-08-20T23:35:50.3064620Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamo_eager_torchbench_inference.csv 2024-08-20T23:35:50.6480251Z llama PASS 2024-08-20T23:35:50.6484455Z llama_v2_7b_16h XFAIL 2024-08-20T23:35:50.6488850Z llava XFAIL 2024-08-20T23:35:50.6492694Z maml XFAIL 2024-08-20T23:35:50.6497201Z maml_omniglot PASS 2024-08-20T23:35:50.6501901Z mnasnet1_0 PASS 2024-08-20T23:35:50.6506423Z mobilenet_v2 PASS 2024-08-20T23:35:50.6510802Z mobilenet_v2_quantized_qat XFAIL 2024-08-20T23:35:50.6515454Z mobilenet_v3_large PASS 2024-08-20T23:35:50.6519824Z moco PASS 2024-08-20T23:35:50.6524063Z moondream XFAIL 2024-08-20T23:35:50.6528433Z nanogpt PASS 2024-08-20T23:35:50.6532920Z nvidia_deeprecommender PASS 2024-08-20T23:35:50.6537363Z opacus_cifar10 PASS 2024-08-20T23:35:50.6542193Z phlippe_densenet PASS 2024-08-20T23:35:50.6546667Z phlippe_resnet PASS 2024-08-20T23:35:50.6551470Z pyhpc_equation_of_state PASS 2024-08-20T23:35:50.6555846Z pyhpc_isoneutral_mixing PASS 2024-08-20T23:35:50.6560211Z pyhpc_turbulent_kinetic_energy PASS 2024-08-20T23:35:50.6564580Z pytorch_CycleGAN_and_pix2pix PASS 2024-08-20T23:35:50.6569001Z pytorch_stargan PASS 2024-08-20T23:35:50.6573004Z pytorch_unet PASS 2024-08-20T23:35:50.6577370Z resnet152 PASS 2024-08-20T23:35:50.6581792Z resnet18 PASS 2024-08-20T23:35:50.6586424Z resnet50 PASS 2024-08-20T23:35:50.6590673Z resnet50_quantized_qat XFAIL 2024-08-20T23:35:50.6595011Z resnext50_32x4d PASS 2024-08-20T23:35:50.6599708Z sam PASS 2024-08-20T23:35:50.6603758Z sam_fast PASS 2024-08-20T23:35:50.6608096Z shufflenet_v2_x1_0 PASS 2024-08-20T23:35:50.6612537Z soft_actor_critic PASS 2024-08-20T23:35:50.6616858Z speech_transformer PASS 2024-08-20T23:35:50.6621304Z squeezenet1_1 PASS 2024-08-20T23:35:50.6625868Z stable_diffusion_text_encoder PASS 2024-08-20T23:35:50.6630212Z stable_diffusion_unet XFAIL 2024-08-20T23:35:50.6634643Z timm_efficientnet PASS 2024-08-20T23:35:50.6639056Z timm_regnet PASS 2024-08-20T23:35:50.6643471Z timm_resnest PASS 2024-08-20T23:35:50.6648220Z timm_vision_transformer PASS 2024-08-20T23:35:50.6652677Z timm_vision_transformer_large XFAIL 2024-08-20T23:35:50.6657074Z timm_vovnet PASS 2024-08-20T23:35:50.6661284Z torch_multimodal_clip PASS 2024-08-20T23:35:50.6666000Z tts_angular PASS 2024-08-20T23:35:50.6670631Z vgg16 PASS 2024-08-20T23:35:50.6674636Z vision_maskrcnn PASS 2024-08-20T23:35:50.6678880Z yolov3 PASS 2024-08-20T23:35:50.7193950Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamo_eager_torchbench_inference.csv 2024-08-20T23:35:51.0549118Z llama PASS 2024-08-20T23:35:51.0553900Z llama_v2_7b_16h PASS 2024-08-20T23:35:51.0558609Z llava PASS 2024-08-20T23:35:51.0563412Z maml PASS 2024-08-20T23:35:51.0568310Z maml_omniglot PASS 2024-08-20T23:35:51.0573236Z mnasnet1_0 PASS 2024-08-20T23:35:51.0578150Z mobilenet_v2 PASS 2024-08-20T23:35:51.0583127Z mobilenet_v2_quantized_qat PASS 2024-08-20T23:35:51.0587994Z mobilenet_v3_large PASS 2024-08-20T23:35:51.0592775Z moco PASS 2024-08-20T23:35:51.0597576Z moondream PASS 2024-08-20T23:35:51.0602339Z nanogpt PASS 2024-08-20T23:35:51.0607251Z nvidia_deeprecommender PASS 2024-08-20T23:35:51.0612035Z opacus_cifar10 PASS 2024-08-20T23:35:51.0616697Z phlippe_densenet PASS 2024-08-20T23:35:51.0621729Z phlippe_resnet PASS 2024-08-20T23:35:51.0626805Z pyhpc_equation_of_state PASS 2024-08-20T23:35:51.0631677Z pyhpc_isoneutral_mixing PASS 2024-08-20T23:35:51.0636639Z pyhpc_turbulent_kinetic_energy PASS 2024-08-20T23:35:51.0641541Z pytorch_CycleGAN_and_pix2pix PASS 2024-08-20T23:35:51.0646439Z pytorch_stargan PASS 2024-08-20T23:35:51.0651892Z pytorch_unet PASS 2024-08-20T23:35:51.0656668Z resnet152 PASS 2024-08-20T23:35:51.0661694Z resnet18 PASS 2024-08-20T23:35:51.0666728Z resnet50 PASS 2024-08-20T23:35:51.0671658Z resnet50_quantized_qat PASS 2024-08-20T23:35:51.0676530Z resnext50_32x4d PASS 2024-08-20T23:35:51.0681476Z sam PASS 2024-08-20T23:35:51.0686380Z sam_fast PASS 2024-08-20T23:35:51.0691295Z shufflenet_v2_x1_0 PASS 2024-08-20T23:35:51.0696202Z soft_actor_critic PASS 2024-08-20T23:35:51.0701119Z speech_transformer PASS 2024-08-20T23:35:51.0706258Z squeezenet1_1 PASS 2024-08-20T23:35:51.0711109Z stable_diffusion_text_encoder PASS 2024-08-20T23:35:51.0716031Z stable_diffusion_unet PASS 2024-08-20T23:35:51.0720940Z timm_efficientnet PASS 2024-08-20T23:35:51.0725798Z timm_regnet PASS 2024-08-20T23:35:51.0730723Z timm_resnest PASS 2024-08-20T23:35:51.0735655Z timm_vision_transformer PASS 2024-08-20T23:35:51.0740532Z timm_vision_transformer_large PASS 2024-08-20T23:35:51.0745569Z timm_vovnet PASS 2024-08-20T23:35:51.0750996Z torch_multimodal_clip PASS 2024-08-20T23:35:51.0755842Z tts_angular PASS 2024-08-20T23:35:51.0760745Z vgg16 PASS 2024-08-20T23:35:51.0765685Z vision_maskrcnn PASS 2024-08-20T23:35:51.0770590Z yolov3 PASS 2024-08-20T23:35:51.1286341Z + test_single_dynamo_benchmark training torchbench 1 --training --amp 2024-08-20T23:35:51.1289715Z ++ pwd 2024-08-20T23:35:51.1293332Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:35:51.1294057Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:35:51.1316851Z + local name=training 2024-08-20T23:35:51.1317817Z + shift 2024-08-20T23:35:51.1318114Z + local suite=torchbench 2024-08-20T23:35:51.1319397Z + shift 2024-08-20T23:35:51.1319733Z + local shard_id=1 2024-08-20T23:35:51.1320038Z + shift 2024-08-20T23:35:51.1323693Z + partition_flags=() 2024-08-20T23:35:51.1324069Z + local partition_flags 2024-08-20T23:35:51.1324569Z + [[ -n 2 ]] 2024-08-20T23:35:51.1324985Z + [[ -n 1 ]] 2024-08-20T23:35:51.1325670Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T23:35:51.1326437Z + [[ dynamo_eager_torchbench == *perf_compare* ]] 2024-08-20T23:35:51.1327203Z + [[ dynamo_eager_torchbench == *perf* ]] 2024-08-20T23:35:51.1327752Z + [[ dynamo_eager_torchbench == *aot_inductor* ]] 2024-08-20T23:35:51.1328312Z + [[ dynamo_eager_torchbench == *_avx2* ]] 2024-08-20T23:35:51.1328792Z + [[ dynamo_eager_torchbench == *_avx512* ]] 2024-08-20T23:35:51.1330596Z + python benchmarks/dynamo/torchbench.py --ci --accuracy --timing --explain --backend eager --device cuda --training --amp --total-partitions 2 --partition-id 1 --output /var/lib/jenkins/workspace/test/test-reports/training_torchbench.csv 2024-08-20T23:35:56.8755424Z 2024-08-20T23:35:57.1783859Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:35:57.1784407Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:35:57.1784838Z cuda train llava 2024-08-20T23:35:57.1791870Z Traceback (most recent call last): 2024-08-20T23:35:57.1792808Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:35:57.1793724Z ) = runner.load_model( 2024-08-20T23:35:57.1794482Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 302, in load_model 2024-08-20T23:35:57.1795229Z benchmark = benchmark_cls( 2024-08-20T23:35:57.1795965Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 24, in __call__ 2024-08-20T23:35:57.1796775Z obj = type.__call__(cls, *args, **kwargs) 2024-08-20T23:35:57.1797635Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llava/__init__.py", line 11, in __init__ 2024-08-20T23:35:57.1798792Z super().__init__(name="llava", test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:35:57.1800086Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/framework/huggingface/model_factory.py", line 81, in __init__ 2024-08-20T23:35:57.1801348Z super().__init__(test=test, device=device, batch_size=batch_size, extra_args=extra_args) 2024-08-20T23:35:57.1802367Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 92, in __init__ 2024-08-20T23:35:57.1803135Z self._skip_by_device_name() 2024-08-20T23:35:57.1803971Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 165, in _skip_by_device_name 2024-08-20T23:35:57.1805245Z raise NotImplementedError(f"The current device {current_device_name} is skipped by its `{self.name}/metadata.yaml`.") 2024-08-20T23:35:57.1806360Z NotImplementedError: The current device NVIDIA A10G is skipped by its `llava/metadata.yaml`. 2024-08-20T23:35:57.1806945Z 2024-08-20T23:35:57.1807066Z model_fail_to_load 2024-08-20T23:36:00.6430350Z 2024-08-20T23:36:01.2703117Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:01.2703585Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:01.2704021Z cuda train maml_omniglot 2024-08-20T23:36:02.0678817Z W0820 23:36:02.067000 37451 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:36:02.7159247Z pass 2024-08-20T23:36:02.7161600Z TIMING: entire_frame_compile:1.1372 backend_compile:0.00032 total_wall_time:1.1372 2024-08-20T23:36:02.7162588Z STATS: call_* op count: 77 | FakeTensorMode.__torch_dispatch__:441 | FakeTensor.__torch_dispatch__:272 2024-08-20T23:36:02.7163510Z Dynamo produced 3 graphs covering 77 ops with 7 graph breaks (5 unique) 2024-08-20T23:36:06.3679595Z 2024-08-20T23:36:08.8693066Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:08.8694059Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:36:08.8694515Z cuda train mnasnet1_0 2024-08-20T23:36:17.2333889Z W0820 23:36:17.232000 37495 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:36:25.0445033Z pass 2024-08-20T23:36:25.0476007Z TIMING: entire_frame_compile:14.3677 backend_compile:0.00201 total_wall_time:14.3677 2024-08-20T23:36:25.0477539Z STATS: call_* op count: 647 | FakeTensorMode.__torch_dispatch__:5042 | FakeTensor.__torch_dispatch__:2864 2024-08-20T23:36:25.0478761Z Dynamo produced 3 graphs covering 647 ops with 7 graph breaks (5 unique) 2024-08-20T23:36:29.0170554Z 2024-08-20T23:36:31.0212650Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:31.0213259Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:36:31.0213755Z cuda train mobilenet_v2 2024-08-20T23:36:39.6889597Z pass 2024-08-20T23:36:39.6939382Z TIMING: entire_frame_compile:6.86511 backend_compile:0.00057 total_wall_time:6.86511 2024-08-20T23:36:39.6941872Z STATS: call_* op count: 157 | FakeTensorMode.__torch_dispatch__:5079 | FakeTensor.__torch_dispatch__:44 2024-08-20T23:36:39.6943103Z Dynamo produced 2 graphs covering 157 ops with 6 graph breaks (5 unique) 2024-08-20T23:36:43.5028748Z 2024-08-20T23:36:45.4533322Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:45.4533849Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:36:45.4534512Z cuda train mobilenet_v2_quantized_qat 2024-08-20T23:36:45.4535213Z Traceback (most recent call last): 2024-08-20T23:36:45.4536013Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2507, in validate_model 2024-08-20T23:36:45.4536806Z self.model_iter_fn(model, example_inputs) 2024-08-20T23:36:45.4537734Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 443, in forward_and_backward_pass 2024-08-20T23:36:45.4538781Z pred = mod(*cloned_inputs) 2024-08-20T23:36:45.4540741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 785, in call_wrapped 2024-08-20T23:36:45.4541990Z return self._wrapped_call(self, *args, **kwargs) 2024-08-20T23:36:45.4543090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 362, in __call__ 2024-08-20T23:36:45.4543980Z raise e 2024-08-20T23:36:45.4544848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 349, in __call__ 2024-08-20T23:36:45.4546099Z return super(self.cls, obj).__call__(*args, **kwargs) # type: ignore[misc] 2024-08-20T23:36:45.4547950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:36:45.4548979Z return self._call_impl(*args, **kwargs) 2024-08-20T23:36:45.4550069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:36:45.4551019Z return forward_call(*args, **kwargs) 2024-08-20T23:36:45.4551526Z File ".3", line 207, in forward 2024-08-20T23:36:45.4552352Z activation_post_process_101 = self.activation_post_process_101(classifier_1); classifier_1 = None 2024-08-20T23:36:45.4553786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:36:45.4554764Z return self._call_impl(*args, **kwargs) 2024-08-20T23:36:45.4555807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:36:45.4556764Z return forward_call(*args, **kwargs) 2024-08-20T23:36:45.4557910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/ao/quantization/fake_quantize.py", line 408, in forward 2024-08-20T23:36:45.4558943Z return torch.fused_moving_avg_obs_fake_quant( 2024-08-20T23:36:45.4559567Z RuntimeError: expected scalar type Float but found Half 2024-08-20T23:36:45.4559983Z 2024-08-20T23:36:45.4560701Z The above exception was the direct cause of the following exception: 2024-08-20T23:36:45.4561222Z 2024-08-20T23:36:45.4561412Z Traceback (most recent call last): 2024-08-20T23:36:45.4562151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:36:45.4562879Z ) = runner.load_model( 2024-08-20T23:36:45.4563624Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 362, in load_model 2024-08-20T23:36:45.4564544Z self.validate_model(model, example_inputs) 2024-08-20T23:36:45.4565607Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2509, in validate_model 2024-08-20T23:36:45.4566502Z raise RuntimeError("Eager run failed") from e 2024-08-20T23:36:45.4566984Z RuntimeError: Eager run failed 2024-08-20T23:36:45.4567300Z 2024-08-20T23:36:45.4567435Z eager_fail_to_run 2024-08-20T23:36:49.0918454Z 2024-08-20T23:36:51.5038096Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:51.5038969Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:36:51.5039882Z cuda train mobilenet_v3_large 2024-08-20T23:37:00.6710199Z W0820 23:37:00.670000 37618 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:37:09.4334695Z pass 2024-08-20T23:37:09.4369942Z TIMING: entire_frame_compile:15.97284 backend_compile:0.00219 total_wall_time:15.97284 2024-08-20T23:37:09.4371013Z STATS: call_* op count: 730 | FakeTensorMode.__torch_dispatch__:5307 | FakeTensor.__torch_dispatch__:3172 2024-08-20T23:37:09.4371974Z Dynamo produced 3 graphs covering 730 ops with 7 graph breaks (5 unique) 2024-08-20T23:37:13.4432266Z 2024-08-20T23:37:15.7654624Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:15.7655091Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:37:15.7658134Z cuda train moco 2024-08-20T23:37:20.3954096Z [rank0]:W0820 23:37:20.394000 37662 torch/_logging/_internal.py:1058] [3/0] Profiler function will be ignored 2024-08-20T23:37:28.4821491Z [rank0]:W0820 23:37:28.481000 37662 torch/_dynamo/backends/distributed.py:89] [4/0_1] Some buckets were extended beyond their requested parameter capacities in order to ensure each subgraph has an output node, required for fx graph partitioning. This can be the case when a subgraph would have only contained nodes performing inplace mutation, and returning no logical outputs. This should not be a problem, unless it results in too few graph partitions for optimal DDP performance. 2024-08-20T23:37:28.5064945Z [rank0]:W0820 23:37:28.505000 37662 torch/_dynamo/backends/distributed.py:106] [4/0_1] DDPOptimizer extended these buckets to ensure per-subgraph output nodes: 2024-08-20T23:37:28.5071059Z [rank0]:W0820 23:37:28.505000 37662 torch/_dynamo/backends/distributed.py:106] [4/0_1] ┌─────────┬─────────────┬────────────────────────┐ 2024-08-20T23:37:28.5072610Z [rank0]:W0820 23:37:28.505000 37662 torch/_dynamo/backends/distributed.py:106] [4/0_1] │ Index │ Extra Ops │ Extra Param Size (b) │ 2024-08-20T23:37:28.5074147Z [rank0]:W0820 23:37:28.505000 37662 torch/_dynamo/backends/distributed.py:106] [4/0_1] ├─────────┼─────────────┼────────────────────────┤ 2024-08-20T23:37:28.5075582Z [rank0]:W0820 23:37:28.505000 37662 torch/_dynamo/backends/distributed.py:106] [4/0_1] │ 0 │ 480 │ 94032128 │ 2024-08-20T23:37:28.5077052Z [rank0]:W0820 23:37:28.505000 37662 torch/_dynamo/backends/distributed.py:106] [4/0_1] └─────────┴─────────────┴────────────────────────┘ 2024-08-20T23:37:36.2185210Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] Graph break from `Tensor.item()`, consider setting: 2024-08-20T23:37:36.2186930Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] torch._dynamo.config.capture_scalar_outputs = True 2024-08-20T23:37:36.2188250Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] or: 2024-08-20T23:37:36.2189958Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-08-20T23:37:36.2191633Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] to include these operations in the captured graph. 2024-08-20T23:37:36.2192742Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] 2024-08-20T23:37:36.2193684Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] Graph break: from user code at: 2024-08-20T23:37:36.2195600Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco/moco/builder.py", line 158, in torch_dynamo_resume_in_forward_at_133 2024-08-20T23:37:36.2197327Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] self._dequeue_and_enqueue(k) 2024-08-20T23:37:36.2199259Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context 2024-08-20T23:37:36.2200798Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] return func(*args, **kwargs) 2024-08-20T23:37:36.2202428Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco/moco/builder.py", line 59, in _dequeue_and_enqueue 2024-08-20T23:37:36.2204037Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] ptr = int(self.queue_ptr) 2024-08-20T23:37:36.2204975Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] 2024-08-20T23:37:36.2205752Z [rank0]:W0820 23:37:36.217000 37662 torch/_dynamo/variables/tensor.py:776] [7/0] 2024-08-20T23:37:40.7833001Z pass 2024-08-20T23:37:40.8197114Z TIMING: entire_frame_compile:20.10735 backend_compile:2.75975 total_wall_time:20.10735 2024-08-20T23:37:40.8198289Z STATS: call_* op count: 889 | FakeTensorMode.__torch_dispatch__:13509 | FakeTensor.__torch_dispatch__:28 2024-08-20T23:37:40.8199307Z Dynamo produced 10 graphs covering 889 ops with 11 graph breaks (8 unique) 2024-08-20T23:37:41.2895197Z [rank0]:[W820 23:37:41.254050430 ProcessGroupNCCL.cpp:1223] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator()) 2024-08-20T23:37:45.4583578Z 2024-08-20T23:37:47.4290297Z loading model: 0it [00:00, ?it/s]number of parameters: 123.69M 2024-08-20T23:37:47.6722279Z num decayed parameter tensors: 50, with 124,354,560 parameters 2024-08-20T23:37:47.6723284Z num non-decayed parameter tensors: 98, with 121,344 parameters 2024-08-20T23:37:47.6723838Z using fused AdamW: True 2024-08-20T23:37:48.1087621Z 2024-08-20T23:37:48.1088286Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:37:48.1088804Z cuda train nanogpt 2024-08-20T23:37:52.8994836Z W0820 23:37:52.898000 37715 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:37:58.8909935Z pass 2024-08-20T23:37:58.9692366Z TIMING: entire_frame_compile:9.12221 backend_compile:0.00176 total_wall_time:9.12221 2024-08-20T23:37:58.9694128Z STATS: call_* op count: 858 | FakeTensorMode.__torch_dispatch__:5065 | FakeTensor.__torch_dispatch__:2756 2024-08-20T23:37:58.9695064Z Dynamo produced 3 graphs covering 858 ops with 7 graph breaks (5 unique) 2024-08-20T23:38:02.7567932Z 2024-08-20T23:38:05.1967794Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:05.1968751Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:38:05.1970126Z cuda train nvidia_deeprecommender 2024-08-20T23:38:06.3904145Z W0820 23:38:06.389000 37759 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:38:07.0907620Z pass 2024-08-20T23:38:07.1123914Z TIMING: entire_frame_compile:0.95613 backend_compile:0.00028 total_wall_time:0.95613 2024-08-20T23:38:07.1126612Z STATS: call_* op count: 70 | FakeTensorMode.__torch_dispatch__:363 | FakeTensor.__torch_dispatch__:236 2024-08-20T23:38:07.1127555Z Dynamo produced 3 graphs covering 70 ops with 7 graph breaks (5 unique) 2024-08-20T23:38:10.6788392Z 2024-08-20T23:38:11.6355656Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:11.6356270Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:11.6356745Z cuda train opacus_cifar10 2024-08-20T23:38:11.6371357Z Traceback (most recent call last): 2024-08-20T23:38:11.6372274Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2507, in validate_model 2024-08-20T23:38:11.6373102Z self.model_iter_fn(model, example_inputs) 2024-08-20T23:38:11.6374178Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 445, in forward_and_backward_pass 2024-08-20T23:38:11.6375364Z self.grad_scaler.scale(loss).backward() 2024-08-20T23:38:11.6376585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_tensor.py", line 522, in backward 2024-08-20T23:38:11.6377360Z torch.autograd.backward( 2024-08-20T23:38:11.6378842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/autograd/__init__.py", line 347, in backward 2024-08-20T23:38:11.6379941Z _engine_run_backward( 2024-08-20T23:38:11.6381176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/autograd/graph.py", line 817, in _engine_run_backward 2024-08-20T23:38:11.6382659Z return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass 2024-08-20T23:38:11.6384013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 98, in __call__ 2024-08-20T23:38:11.6385184Z return self.hook(module, *args, **kwargs) 2024-08-20T23:38:11.6386467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/opacus/grad_sample/grad_sample_module.py", line 328, in capture_backprops_hook 2024-08-20T23:38:11.6387526Z activations, backprops = self.rearrange_grad_samples( 2024-08-20T23:38:11.6388702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/opacus/grad_sample/grad_sample_module.py", line 384, in rearrange_grad_samples 2024-08-20T23:38:11.6389661Z raise ValueError( 2024-08-20T23:38:11.6390519Z ValueError: No activations detected for , run forward after add_hooks(model) 2024-08-20T23:38:11.6391222Z 2024-08-20T23:38:11.6391542Z The above exception was the direct cause of the following exception: 2024-08-20T23:38:11.6391998Z 2024-08-20T23:38:11.6392157Z Traceback (most recent call last): 2024-08-20T23:38:11.6392826Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:38:11.6393507Z ) = runner.load_model( 2024-08-20T23:38:11.6394192Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 362, in load_model 2024-08-20T23:38:11.6394980Z self.validate_model(model, example_inputs) 2024-08-20T23:38:11.6395751Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2509, in validate_model 2024-08-20T23:38:11.6396552Z raise RuntimeError("Eager run failed") from e 2024-08-20T23:38:11.6397043Z RuntimeError: Eager run failed 2024-08-20T23:38:11.6397307Z 2024-08-20T23:38:11.6397436Z eager_fail_to_run 2024-08-20T23:38:15.2087699Z 2024-08-20T23:38:16.0311899Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:16.0312387Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:16.0312847Z cuda train phlippe_densenet 2024-08-20T23:38:23.8551554Z pass 2024-08-20T23:38:23.8557110Z TIMING: entire_frame_compile:7.14297 backend_compile:0.00059 total_wall_time:7.14297 2024-08-20T23:38:23.8558281Z STATS: call_* op count: 190 | FakeTensorMode.__torch_dispatch__:5196 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:38:23.8559207Z Dynamo produced 2 graphs covering 190 ops with 6 graph breaks (5 unique) 2024-08-20T23:38:27.5984797Z 2024-08-20T23:38:28.3065951Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:28.3066565Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:28.3067041Z cuda train phlippe_resnet 2024-08-20T23:38:31.0812479Z pass 2024-08-20T23:38:31.0813323Z TIMING: entire_frame_compile:2.37227 backend_compile:0.00029 total_wall_time:2.37227 2024-08-20T23:38:31.0814349Z STATS: call_* op count: 75 | FakeTensorMode.__torch_dispatch__:1976 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:38:31.0815259Z Dynamo produced 2 graphs covering 75 ops with 6 graph breaks (5 unique) 2024-08-20T23:38:34.7413443Z 2024-08-20T23:38:36.0223661Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:36.0224162Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:38:36.0224616Z cuda train pytorch_CycleGAN_and_pix2pix 2024-08-20T23:38:47.1081468Z pass 2024-08-20T23:38:47.1172046Z TIMING: entire_frame_compile:2.19935 backend_compile:0.0002 total_wall_time:2.19935 2024-08-20T23:38:47.1173208Z STATS: call_* op count: 95 | FakeTensorMode.__torch_dispatch__:2066 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:38:47.1174134Z Dynamo produced 2 graphs covering 95 ops with 6 graph breaks (5 unique) 2024-08-20T23:38:50.8176980Z 2024-08-20T23:38:52.3048751Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:52.3049374Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:38:52.3049847Z cuda train pytorch_stargan 2024-08-20T23:39:20.7241677Z pass 2024-08-20T23:39:20.7443123Z TIMING: entire_frame_compile:2.22765 backend_compile:0.00025 total_wall_time:2.22765 2024-08-20T23:39:20.7445323Z STATS: call_* op count: 60 | FakeTensorMode.__torch_dispatch__:2546 | FakeTensor.__torch_dispatch__:26 2024-08-20T23:39:20.7446963Z Dynamo produced 2 graphs covering 60 ops with 6 graph breaks (5 unique) 2024-08-20T23:39:24.4272047Z 2024-08-20T23:39:25.3641854Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:39:25.3642485Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:39:25.3643021Z cuda train pytorch_unet 2024-08-20T23:39:47.5307607Z W0820 23:39:47.530000 38015 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:39:50.9331653Z pass_due_to_skip 2024-08-20T23:39:51.1330362Z TIMING: entire_frame_compile:6.03542 backend_compile:0.00096 total_wall_time:6.03542 2024-08-20T23:39:51.1332911Z STATS: call_* op count: 314 | FakeTensorMode.__torch_dispatch__:2581 | FakeTensor.__torch_dispatch__:1352 2024-08-20T23:39:51.1334544Z Dynamo produced 3 graphs covering 314 ops with 7 graph breaks (5 unique) 2024-08-20T23:39:54.8622228Z 2024-08-20T23:39:57.2823806Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:39:57.2824465Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:39:57.2825001Z cuda train resnet152 2024-08-20T23:40:21.6152626Z W0820 23:40:21.614000 38052 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:40:44.9819840Z pass 2024-08-20T23:40:45.0255133Z TIMING: entire_frame_compile:41.48886 backend_compile:0.00684 total_wall_time:41.48886 2024-08-20T23:40:45.0256187Z STATS: call_* op count: 1937 | FakeTensorMode.__torch_dispatch__:14671 | FakeTensor.__torch_dispatch__:8426 2024-08-20T23:40:45.0257618Z Dynamo produced 3 graphs covering 1937 ops with 7 graph breaks (5 unique) 2024-08-20T23:40:49.6516432Z 2024-08-20T23:40:50.9099423Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:40:50.9100064Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:40:50.9100675Z cuda train resnet18 2024-08-20T23:40:54.8352662Z pass 2024-08-20T23:40:54.8430152Z TIMING: entire_frame_compile:2.33025 backend_compile:0.00028 total_wall_time:2.33025 2024-08-20T23:40:54.8431581Z STATS: call_* op count: 73 | FakeTensorMode.__torch_dispatch__:2012 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:40:54.8432499Z Dynamo produced 2 graphs covering 73 ops with 6 graph breaks (5 unique) 2024-08-20T23:40:58.5497712Z 2024-08-20T23:41:00.3733901Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:41:00.3734372Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:41:11.2687697Z cuda train resnet50 2024-08-20T23:41:11.2688195Z pass 2024-08-20T23:41:11.2879872Z TIMING: entire_frame_compile:6.54372 backend_compile:0.0006 total_wall_time:6.54372 2024-08-20T23:41:11.2882145Z STATS: call_* op count: 179 | FakeTensorMode.__torch_dispatch__:5134 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:41:11.2884243Z Dynamo produced 2 graphs covering 179 ops with 6 graph breaks (5 unique) 2024-08-20T23:41:15.0934255Z 2024-08-20T23:41:17.1583771Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:41:17.1584484Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:41:17.1585052Z cuda train resnet50_quantized_qat 2024-08-20T23:41:17.1585533Z Traceback (most recent call last): 2024-08-20T23:41:17.1586498Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2507, in validate_model 2024-08-20T23:41:17.1587484Z self.model_iter_fn(model, example_inputs) 2024-08-20T23:41:17.1588332Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 443, in forward_and_backward_pass 2024-08-20T23:41:17.1589724Z pred = mod(*cloned_inputs) 2024-08-20T23:41:17.1591105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 785, in call_wrapped 2024-08-20T23:41:17.1592009Z return self._wrapped_call(self, *args, **kwargs) 2024-08-20T23:41:17.1593103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 362, in __call__ 2024-08-20T23:41:17.1594172Z raise e 2024-08-20T23:41:17.1595071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/fx/graph_module.py", line 349, in __call__ 2024-08-20T23:41:17.1596346Z return super(self.cls, obj).__call__(*args, **kwargs) # type: ignore[misc] 2024-08-20T23:41:17.1597928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:41:17.1599049Z return self._call_impl(*args, **kwargs) 2024-08-20T23:41:17.1600021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:41:17.1600844Z return forward_call(*args, **kwargs) 2024-08-20T23:41:17.1601332Z File ".3", line 167, in forward 2024-08-20T23:41:17.1601996Z activation_post_process_73 = self.activation_post_process_73(fc); fc = None 2024-08-20T23:41:17.1603193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:41:17.1604132Z return self._call_impl(*args, **kwargs) 2024-08-20T23:41:17.1605095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:41:17.1605958Z return forward_call(*args, **kwargs) 2024-08-20T23:41:17.1606960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/ao/quantization/fake_quantize.py", line 408, in forward 2024-08-20T23:41:17.1607906Z return torch.fused_moving_avg_obs_fake_quant( 2024-08-20T23:41:17.1608471Z RuntimeError: expected scalar type Float but found Half 2024-08-20T23:41:17.1608877Z 2024-08-20T23:41:17.1609182Z The above exception was the direct cause of the following exception: 2024-08-20T23:41:17.1609653Z 2024-08-20T23:41:17.1609803Z Traceback (most recent call last): 2024-08-20T23:41:17.1610475Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:41:17.1611151Z ) = runner.load_model( 2024-08-20T23:41:17.1611838Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 362, in load_model 2024-08-20T23:41:17.1613006Z self.validate_model(model, example_inputs) 2024-08-20T23:41:17.1613794Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2509, in validate_model 2024-08-20T23:41:17.1614593Z raise RuntimeError("Eager run failed") from e 2024-08-20T23:41:17.1615072Z RuntimeError: Eager run failed 2024-08-20T23:41:17.1615340Z 2024-08-20T23:41:17.1615462Z eager_fail_to_run 2024-08-20T23:41:20.7785195Z 2024-08-20T23:41:22.7387026Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:41:22.7388523Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:41:22.7389395Z cuda train resnext50_32x4d 2024-08-20T23:41:35.0499306Z W0820 23:41:35.049000 38219 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:41:42.9762683Z pass 2024-08-20T23:41:42.9928416Z TIMING: entire_frame_compile:14.23815 backend_compile:0.00205 total_wall_time:14.23815 2024-08-20T23:41:42.9929431Z STATS: call_* op count: 679 | FakeTensorMode.__torch_dispatch__:5143 | FakeTensor.__torch_dispatch__:2918 2024-08-20T23:41:42.9932654Z Dynamo produced 3 graphs covering 679 ops with 7 graph breaks (5 unique) 2024-08-20T23:41:46.9161024Z 2024-08-20T23:41:54.4827884Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:41:54.4828504Z loading model: 0it [00:07, ?it/s] 2024-08-20T23:41:54.4829034Z cuda train sam 2024-08-20T23:41:54.4837487Z Traceback (most recent call last): 2024-08-20T23:41:54.4838445Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2507, in validate_model 2024-08-20T23:41:54.4839228Z self.model_iter_fn(model, example_inputs) 2024-08-20T23:41:54.4840103Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 445, in forward_and_backward_pass 2024-08-20T23:41:54.4841103Z self.grad_scaler.scale(loss).backward() 2024-08-20T23:41:54.4842476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_tensor.py", line 522, in backward 2024-08-20T23:41:54.4843475Z torch.autograd.backward( 2024-08-20T23:41:54.4844370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/autograd/__init__.py", line 347, in backward 2024-08-20T23:41:54.4845188Z _engine_run_backward( 2024-08-20T23:41:54.4846121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/autograd/graph.py", line 817, in _engine_run_backward 2024-08-20T23:41:54.4848312Z return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass 2024-08-20T23:41:54.4849327Z RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn 2024-08-20T23:41:54.4849845Z 2024-08-20T23:41:54.4850144Z The above exception was the direct cause of the following exception: 2024-08-20T23:41:54.4850587Z 2024-08-20T23:41:54.4850740Z Traceback (most recent call last): 2024-08-20T23:41:54.4851407Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:41:54.4852067Z ) = runner.load_model( 2024-08-20T23:41:54.4852727Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 362, in load_model 2024-08-20T23:41:54.4853487Z self.validate_model(model, example_inputs) 2024-08-20T23:41:54.4854243Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2509, in validate_model 2024-08-20T23:41:54.4855001Z raise RuntimeError("Eager run failed") from e 2024-08-20T23:41:54.4855471Z RuntimeError: Eager run failed 2024-08-20T23:41:54.4855733Z 2024-08-20T23:41:54.4855856Z eager_fail_to_run 2024-08-20T23:41:58.1182701Z 2024-08-20T23:41:59.4205973Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:41:59.4207159Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:41:59.4208161Z cuda train shufflenet_v2_x1_0 2024-08-20T23:42:07.5673606Z pass 2024-08-20T23:42:07.5684083Z TIMING: entire_frame_compile:7.17111 backend_compile:0.00063 total_wall_time:7.17111 2024-08-20T23:42:07.5685548Z STATS: call_* op count: 275 | FakeTensorMode.__torch_dispatch__:5965 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:42:07.5686479Z Dynamo produced 2 graphs covering 275 ops with 6 graph breaks (5 unique) 2024-08-20T23:42:11.4315728Z 2024-08-20T23:42:12.2671925Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:12.2672468Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:12.2672909Z cuda train soft_actor_critic 2024-08-20T23:42:12.8332327Z W0820 23:42:12.832000 38364 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:42:13.1382623Z pass 2024-08-20T23:42:13.1385315Z TIMING: entire_frame_compile:0.62651 backend_compile:0.00027 total_wall_time:0.62651 2024-08-20T23:42:13.1386411Z STATS: call_* op count: 56 | FakeTensorMode.__torch_dispatch__:271 | FakeTensor.__torch_dispatch__:127 2024-08-20T23:42:13.1387416Z Dynamo produced 4 graphs covering 56 ops with 6 graph breaks (5 unique) 2024-08-20T23:42:16.7490786Z 2024-08-20T23:42:18.0416426Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:18.0416944Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:42:18.0418888Z cuda train speech_transformer 2024-08-20T23:42:25.3000788Z pass 2024-08-20T23:42:25.3474308Z TIMING: entire_frame_compile:4.69388 backend_compile:0.00117 total_wall_time:4.69388 2024-08-20T23:42:25.3475497Z STATS: call_* op count: 771 | FakeTensorMode.__torch_dispatch__:4790 | FakeTensor.__torch_dispatch__:66 2024-08-20T23:42:25.3476470Z Dynamo produced 10 graphs covering 771 ops with 16 graph breaks (7 unique) 2024-08-20T23:42:29.0742687Z 2024-08-20T23:42:30.2206918Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:30.2207387Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:42:30.2207842Z cuda train squeezenet1_1 2024-08-20T23:42:32.8012449Z pass 2024-08-20T23:42:32.8026658Z TIMING: entire_frame_compile:1.43919 backend_compile:0.0002 total_wall_time:1.43919 2024-08-20T23:42:32.8027697Z STATS: call_* op count: 70 | FakeTensorMode.__torch_dispatch__:1006 | FakeTensor.__torch_dispatch__:26 2024-08-20T23:42:32.8028633Z Dynamo produced 2 graphs covering 70 ops with 6 graph breaks (5 unique) 2024-08-20T23:42:36.4963619Z 2024-08-20T23:42:37.7622706Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:37.7623155Z 2024-08-20T23:42:38.8126331Z Loading pipeline components...: 0% 0/6 [00:00 will be ignored 2024-08-20T23:43:29.4256398Z pass 2024-08-20T23:43:29.4302954Z TIMING: entire_frame_compile:19.30084 backend_compile:0.00274 total_wall_time:19.30084 2024-08-20T23:43:29.4304154Z STATS: call_* op count: 899 | FakeTensorMode.__torch_dispatch__:6100 | FakeTensor.__torch_dispatch__:3892 2024-08-20T23:43:29.4306392Z Dynamo produced 3 graphs covering 899 ops with 7 graph breaks (5 unique) 2024-08-20T23:43:33.6711472Z 2024-08-20T23:43:36.4585549Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:43:36.4586157Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:43:36.4586641Z cuda train timm_regnet 2024-08-20T23:43:57.3508413Z pass 2024-08-20T23:43:57.3934016Z TIMING: entire_frame_compile:10.92152 backend_compile:0.00089 total_wall_time:10.92152 2024-08-20T23:43:57.3935184Z STATS: call_* op count: 323 | FakeTensorMode.__torch_dispatch__:7384 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:43:57.3936412Z Dynamo produced 2 graphs covering 323 ops with 6 graph breaks (5 unique) 2024-08-20T23:44:01.3409812Z 2024-08-20T23:44:03.3349258Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:44:03.3349862Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:44:03.3351983Z cuda train timm_resnest 2024-08-20T23:44:10.5925698Z W0820 23:44:10.591000 38680 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:44:14.7579931Z pass 2024-08-20T23:44:14.7664135Z TIMING: entire_frame_compile:7.25389 backend_compile:0.0011 total_wall_time:7.25389 2024-08-20T23:44:14.7665338Z STATS: call_* op count: 401 | FakeTensorMode.__torch_dispatch__:2901 | FakeTensor.__torch_dispatch__:1514 2024-08-20T23:44:14.7666268Z Dynamo produced 3 graphs covering 401 ops with 7 graph breaks (5 unique) 2024-08-20T23:44:18.7112290Z 2024-08-20T23:44:20.2549296Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:44:20.2549874Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:44:20.2550635Z cuda train timm_vision_transformer 2024-08-20T23:44:25.1470947Z pass 2024-08-20T23:44:25.1631946Z TIMING: entire_frame_compile:3.61648 backend_compile:0.00045 total_wall_time:3.61648 2024-08-20T23:44:25.1633035Z STATS: call_* op count: 267 | FakeTensorMode.__torch_dispatch__:4115 | FakeTensor.__torch_dispatch__:92 2024-08-20T23:44:25.1634002Z Dynamo produced 2 graphs covering 267 ops with 6 graph breaks (5 unique) 2024-08-20T23:44:28.9987438Z 2024-08-20T23:44:41.5513269Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:44:41.5513866Z loading model: 0it [00:12, ?it/s] 2024-08-20T23:44:41.5514402Z cuda train timm_vision_transformer_large 2024-08-20T23:44:41.5516582Z pass_due_to_skip 2024-08-20T23:44:41.8198272Z TIMING: 2024-08-20T23:44:41.8198674Z STATS: call_* op count: 0 2024-08-20T23:44:41.8199371Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:44:45.5189622Z 2024-08-20T23:44:47.5598005Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:44:47.5598610Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:44:47.5599050Z cuda train timm_vovnet 2024-08-20T23:44:59.2635538Z pass 2024-08-20T23:44:59.2812291Z TIMING: entire_frame_compile:5.50356 backend_compile:0.00045 total_wall_time:5.50356 2024-08-20T23:44:59.2813684Z STATS: call_* op count: 134 | FakeTensorMode.__torch_dispatch__:3828 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:44:59.2815480Z Dynamo produced 2 graphs covering 134 ops with 6 graph breaks (5 unique) 2024-08-20T23:45:03.1972096Z 2024-08-20T23:45:05.2003910Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:45:05.2004432Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:45:05.2004883Z cuda train torch_multimodal_clip 2024-08-20T23:45:17.7908316Z W0820 23:45:17.790000 38867 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:45:29.8322680Z pass 2024-08-20T23:45:29.9248859Z TIMING: entire_frame_compile:19.79656 backend_compile:0.0037 total_wall_time:19.79656 2024-08-20T23:45:29.9249868Z STATS: call_* op count: 1404 | FakeTensorMode.__torch_dispatch__:10581 | FakeTensor.__torch_dispatch__:5624 2024-08-20T23:45:29.9250805Z Dynamo produced 3 graphs covering 1404 ops with 7 graph breaks (5 unique) 2024-08-20T23:45:34.0294831Z 2024-08-20T23:45:34.6978031Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:45:34.6978555Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:45:34.6978995Z cuda train tts_angular 2024-08-20T23:45:35.3275318Z W0820 23:45:35.326000 38911 torch/_logging/_internal.py:1058] [10/0] Profiler function will be ignored 2024-08-20T23:45:35.9933272Z pass 2024-08-20T23:45:36.0004968Z TIMING: entire_frame_compile:0.82822 backend_compile:0.00037 total_wall_time:0.82822 2024-08-20T23:45:36.0006063Z STATS: call_* op count: 70 | FakeTensor.__torch_dispatch__:302 | FakeTensorMode.__torch_dispatch__:69 2024-08-20T23:45:36.0006941Z Dynamo produced 5 graphs covering 70 ops with 9 graph breaks (6 unique) 2024-08-20T23:45:39.6115387Z 2024-08-20T23:45:42.2303735Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:45:42.2304350Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:45:42.2304824Z cuda train vgg16 2024-08-20T23:46:00.5307792Z pass 2024-08-20T23:46:00.5575259Z TIMING: entire_frame_compile:0.90179 backend_compile:0.00017 total_wall_time:0.90179 2024-08-20T23:46:00.5576426Z STATS: call_* op count: 44 | FakeTensorMode.__torch_dispatch__:692 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:46:00.5577334Z Dynamo produced 2 graphs covering 44 ops with 6 graph breaks (5 unique) 2024-08-20T23:46:04.2177935Z 2024-08-20T23:46:07.6556790Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:46:07.6557270Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:46:07.6557767Z cuda train vision_maskrcnn 2024-08-20T23:46:27.2051847Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] Graph break from `Tensor.item()`, consider setting: 2024-08-20T23:46:27.2053208Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] torch._dynamo.config.capture_scalar_outputs = True 2024-08-20T23:46:27.2054249Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] or: 2024-08-20T23:46:27.2055512Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-08-20T23:46:27.2057189Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] to include these operations in the captured graph. 2024-08-20T23:46:27.2058355Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] 2024-08-20T23:46:27.2059620Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] Graph break: from user code at: 2024-08-20T23:46:27.2062278Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/models/detection/generalized_rcnn.py", line 105, in torch_dynamo_resume_in_forward_at_104 2024-08-20T23:46:27.2064660Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] detections, detector_losses = self.roi_heads(features, proposals, images.image_sizes, targets) 2024-08-20T23:46:27.2066669Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/models/detection/roi_heads.py", line 761, in forward 2024-08-20T23:46:27.2068804Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] box_features = self.box_roi_pool(features, proposals, image_shapes) 2024-08-20T23:46:27.2070733Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/poolers.py", line 310, in forward 2024-08-20T23:46:27.2072251Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] self.scales, self.map_levels = _setup_scales( 2024-08-20T23:46:27.2073936Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/poolers.py", line 122, in _setup_scales 2024-08-20T23:46:27.2075586Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] scales = [_infer_scale(feat, original_input_shape) for feat in features] 2024-08-20T23:46:27.2077382Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/poolers.py", line 122, in 2024-08-20T23:46:27.2079018Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] scales = [_infer_scale(feat, original_input_shape) for feat in features] 2024-08-20T23:46:27.2080807Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/poolers.py", line 104, in _infer_scale 2024-08-20T23:46:27.2082398Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] scale = 2 ** float(torch.tensor(approx_scale).log2().round()) 2024-08-20T23:46:27.2083425Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] 2024-08-20T23:46:27.2084173Z W0820 23:46:27.204000 38999 torch/_dynamo/variables/tensor.py:776] [18/0] 2024-08-20T23:46:55.1409023Z W0820 23:46:55.140000 38999 torch/_dynamo/convert_frame.py:831] [30/8] torch._dynamo hit config.cache_size_limit (8) 2024-08-20T23:46:55.1411130Z W0820 23:46:55.140000 38999 torch/_dynamo/convert_frame.py:831] [30/8] function: 'roi_align' (/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/roi_align.py:186) 2024-08-20T23:46:55.1413109Z W0820 23:46:55.140000 38999 torch/_dynamo/convert_frame.py:831] [30/8] last reason: 30/0: tensor 'L['boxes']' size mismatch at index 0. expected 648, actual 619 2024-08-20T23:46:55.1414579Z W0820 23:46:55.140000 38999 torch/_dynamo/convert_frame.py:831] [30/8] To log all recompilation reasons, use TORCH_LOGS="recompiles". 2024-08-20T23:46:55.1416169Z W0820 23:46:55.140000 38999 torch/_dynamo/convert_frame.py:831] [30/8] To diagnose recompilation issues, see https://pytorch.org/docs/main/torch.compiler_troubleshooting.html. 2024-08-20T23:47:01.8676849Z pass 2024-08-20T23:47:01.9304974Z TIMING: entire_frame_compile:46.04155 backend_compile:0.00408 total_wall_time:46.04155 2024-08-20T23:47:01.9306700Z STATS: call_* op count: 2436 | FakeTensorMode.__torch_dispatch__:29324 | FakeTensor.__torch_dispatch__:622 | attempt fast:501 | slow no contiguity match:38 | fast is_contiguous:349 | slow both tensors nontrivially broadcast:114 2024-08-20T23:47:01.9308204Z Dynamo produced 51 graphs covering 2436 ops with 33 graph breaks (8 unique) 2024-08-20T23:47:06.4937358Z 2024-08-20T23:47:09.3609571Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:47:09.3610182Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:47:09.3610761Z cuda train yolov3 2024-08-20T23:47:37.9451140Z W0820 23:47:37.944000 39045 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:47:59.1102716Z pass 2024-08-20T23:47:59.1642535Z TIMING: entire_frame_compile:31.41061 backend_compile:0.00395 total_wall_time:31.41061 2024-08-20T23:47:59.1644005Z STATS: call_* op count: 1377 | FakeTensorMode.__torch_dispatch__:15193 | FakeTensor.__torch_dispatch__:4100 2024-08-20T23:47:59.1647346Z Dynamo produced 4 graphs covering 1377 ops with 8 graph breaks (5 unique) 2024-08-20T23:48:00.7058493Z accuracy pass_rate=78.95% 2024-08-20T23:48:00.7061041Z calls_captured gmean=0.00x mean=431.184x 2024-08-20T23:48:00.7064707Z unique_graphs gmean=0.00x mean=3.816x 2024-08-20T23:48:00.7068177Z graph_breaks gmean=0.00x mean=6.395x 2024-08-20T23:48:00.7071756Z unique_graph_breaks gmean=0.00x mean=4.316x 2024-08-20T23:48:00.7074494Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T23:48:00.7077864Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T23:48:00.7081542Z cudagraph_skips gmean=0.00x mean=0.000x 2024-08-20T23:48:01.2983171Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamo_eager_torchbench_training.csv 2024-08-20T23:48:01.6341674Z llava XFAIL 2024-08-20T23:48:01.6346403Z maml_omniglot PASS 2024-08-20T23:48:01.6351568Z mnasnet1_0 PASS 2024-08-20T23:48:01.6355995Z mobilenet_v2 PASS 2024-08-20T23:48:01.6360415Z mobilenet_v2_quantized_qat XFAIL 2024-08-20T23:48:01.6364934Z mobilenet_v3_large PASS 2024-08-20T23:48:01.6369236Z moco PASS 2024-08-20T23:48:01.6374246Z nanogpt PASS 2024-08-20T23:48:01.6378667Z nvidia_deeprecommender PASS 2024-08-20T23:48:01.6383090Z opacus_cifar10 XFAIL 2024-08-20T23:48:01.6387419Z phlippe_densenet PASS 2024-08-20T23:48:01.6391584Z phlippe_resnet PASS 2024-08-20T23:48:01.6395902Z pytorch_CycleGAN_and_pix2pix PASS 2024-08-20T23:48:01.6400242Z pytorch_stargan PASS 2024-08-20T23:48:01.6404499Z pytorch_unet XFAIL 2024-08-20T23:48:01.6409238Z resnet152 PASS 2024-08-20T23:48:01.6413698Z resnet18 PASS 2024-08-20T23:48:01.6418020Z resnet50 PASS 2024-08-20T23:48:01.6422447Z resnet50_quantized_qat XFAIL 2024-08-20T23:48:01.6426891Z resnext50_32x4d PASS 2024-08-20T23:48:01.6431354Z sam XFAIL 2024-08-20T23:48:01.6435707Z shufflenet_v2_x1_0 PASS 2024-08-20T23:48:01.6440106Z soft_actor_critic PASS 2024-08-20T23:48:01.6444212Z speech_transformer PASS 2024-08-20T23:48:01.6449154Z squeezenet1_1 PASS 2024-08-20T23:48:01.6453319Z stable_diffusion_text_encoder PASS 2024-08-20T23:48:01.6457710Z stable_diffusion_unet XFAIL 2024-08-20T23:48:01.6462289Z timm_efficientnet PASS 2024-08-20T23:48:01.6466566Z timm_regnet PASS 2024-08-20T23:48:01.6470912Z timm_resnest PASS 2024-08-20T23:48:01.6475316Z timm_vision_transformer PASS 2024-08-20T23:48:01.6479714Z timm_vision_transformer_large XFAIL 2024-08-20T23:48:01.6483988Z timm_vovnet PASS 2024-08-20T23:48:01.6488358Z torch_multimodal_clip PASS 2024-08-20T23:48:01.6492677Z tts_angular PASS 2024-08-20T23:48:01.6497100Z vgg16 PASS 2024-08-20T23:48:01.6501610Z vision_maskrcnn PASS 2024-08-20T23:48:01.6506056Z yolov3 PASS 2024-08-20T23:48:01.7034864Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamo_eager_torchbench_training.csv 2024-08-20T23:48:02.0387795Z llava PASS 2024-08-20T23:48:02.0391670Z maml_omniglot PASS 2024-08-20T23:48:02.0396355Z mnasnet1_0 PASS 2024-08-20T23:48:02.0401059Z mobilenet_v2 PASS 2024-08-20T23:48:02.0405589Z mobilenet_v2_quantized_qat PASS 2024-08-20T23:48:02.0410060Z mobilenet_v3_large PASS 2024-08-20T23:48:02.0414517Z moco PASS 2024-08-20T23:48:02.0418833Z nanogpt PASS 2024-08-20T23:48:02.0423843Z nvidia_deeprecommender PASS 2024-08-20T23:48:02.0428473Z opacus_cifar10 PASS 2024-08-20T23:48:02.0432682Z phlippe_densenet PASS 2024-08-20T23:48:02.0437055Z phlippe_resnet PASS 2024-08-20T23:48:02.0441547Z pytorch_CycleGAN_and_pix2pix PASS 2024-08-20T23:48:02.0445958Z pytorch_stargan PASS 2024-08-20T23:48:02.0450693Z pytorch_unet PASS 2024-08-20T23:48:02.0455120Z resnet152 PASS 2024-08-20T23:48:02.0459590Z resnet18 PASS 2024-08-20T23:48:02.0464167Z resnet50 PASS 2024-08-20T23:48:02.0468599Z resnet50_quantized_qat PASS 2024-08-20T23:48:02.0472948Z resnext50_32x4d PASS 2024-08-20T23:48:02.0477459Z sam PASS 2024-08-20T23:48:02.0481856Z shufflenet_v2_x1_0 PASS 2024-08-20T23:48:02.0486472Z soft_actor_critic PASS 2024-08-20T23:48:02.0490566Z speech_transformer PASS 2024-08-20T23:48:02.0495077Z squeezenet1_1 PASS 2024-08-20T23:48:02.0499400Z stable_diffusion_text_encoder PASS 2024-08-20T23:48:02.0504068Z stable_diffusion_unet PASS 2024-08-20T23:48:02.0508428Z timm_efficientnet PASS 2024-08-20T23:48:02.0512819Z timm_regnet PASS 2024-08-20T23:48:02.0517199Z timm_resnest PASS 2024-08-20T23:48:02.0521610Z timm_vision_transformer PASS 2024-08-20T23:48:02.0526052Z timm_vision_transformer_large PASS 2024-08-20T23:48:02.0530469Z timm_vovnet PASS 2024-08-20T23:48:02.0534888Z torch_multimodal_clip PASS 2024-08-20T23:48:02.0539305Z tts_angular PASS 2024-08-20T23:48:02.0543834Z vgg16 PASS 2024-08-20T23:48:02.0548650Z vision_maskrcnn PASS 2024-08-20T23:48:02.0553040Z yolov3 PASS 2024-08-20T23:48:02.1065625Z + cleanup_workspace 2024-08-20T23:48:02.1066880Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T23:48:02.1068216Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T23:48:02.1069468Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T23:48:02.1070130Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T23:48:02.1070968Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T23:48:02.1071879Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T23:48:02.1072601Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T23:48:03.1339044Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T23:48:03.1339582Z cat test/**/*_toprint.log || true 2024-08-20T23:48:03.1353695Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:48:03.1354212Z env: 2024-08-20T23:48:03.1354495Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:03.1354967Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:03.1355717Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:03.1356362Z ##[endgroup] 2024-08-20T23:48:03.1462946Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T23:48:03.1496554Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:48:03.1497210Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:48:03.1505616Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:48:03.1506129Z env: 2024-08-20T23:48:03.1506399Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:03.1506857Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:03.1507591Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:03.1508246Z MONITOR_SCRIPT_PID: 375230 2024-08-20T23:48:03.1508599Z ##[endgroup] 2024-08-20T23:48:03.1712164Z Prepare all required actions 2024-08-20T23:48:03.1712618Z Getting action download info 2024-08-20T23:48:03.3593688Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T23:48:03.5584435Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T23:48:03.5584915Z with: 2024-08-20T23:48:03.5585482Z file-suffix: test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067 2024-08-20T23:48:03.5586203Z s3-bucket: gha-artifacts 2024-08-20T23:48:03.5586544Z env: 2024-08-20T23:48:03.5586821Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:03.5587287Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:03.5588017Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:03.5588644Z ##[endgroup] 2024-08-20T23:48:03.5615134Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T23:48:03.5615806Z # Remove any previous test jsons if they exist 2024-08-20T23:48:03.5616334Z rm -f test-jsons-*.zip 2024-08-20T23:48:03.5616871Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T23:48:03.5625941Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:48:03.5626439Z env: 2024-08-20T23:48:03.5626720Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:03.5627187Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:03.5627925Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:03.5628902Z FILE_SUFFIX: test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067 2024-08-20T23:48:03.5629579Z ##[endgroup] 2024-08-20T23:48:03.5987236Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T23:48:03.6016185Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T23:48:03.6016893Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T23:48:03.6023293Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T23:48:03.6028856Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T23:48:03.6062048Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T23:48:03.6062714Z # Remove any previous test reports if they exist 2024-08-20T23:48:03.6063261Z rm -f test-reports-*.zip 2024-08-20T23:48:03.6063868Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T23:48:03.6072888Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:48:03.6073390Z env: 2024-08-20T23:48:03.6073669Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:03.6074132Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:03.6074860Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:03.6075768Z FILE_SUFFIX: test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067 2024-08-20T23:48:03.6076442Z ##[endgroup] 2024-08-20T23:48:03.6336189Z adding: test/test-reports/inference_torchbench.csv (deflated 69%) 2024-08-20T23:48:03.6338499Z adding: test/test-reports/inference_torchbench_graph_breaks.csv (deflated 92%) 2024-08-20T23:48:03.6339713Z adding: test/test-reports/inference_torchbench_graph_break_deduped.csv (deflated 76%) 2024-08-20T23:48:03.6340612Z adding: test/test-reports/training_torchbench.csv (deflated 67%) 2024-08-20T23:48:03.6349106Z adding: test/test-reports/training_torchbench_graph_breaks.csv (deflated 96%) 2024-08-20T23:48:03.6350542Z adding: test/test-reports/training_torchbench_graph_break_deduped.csv (deflated 76%) 2024-08-20T23:48:03.6384527Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T23:48:03.6385173Z # Remove any previous usage logs if they exist 2024-08-20T23:48:03.6385688Z rm -f logs-*.zip 2024-08-20T23:48:03.6386591Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T23:48:03.6387413Z # so check to see if the file exists first 2024-08-20T23:48:03.6387939Z if [ -f 'usage_log.txt' ]; then 2024-08-20T23:48:03.6388483Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T23:48:03.6388986Z fi 2024-08-20T23:48:03.6389360Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T23:48:03.6389957Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T23:48:03.6390472Z fi 2024-08-20T23:48:03.6399137Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:48:03.6399645Z env: 2024-08-20T23:48:03.6399926Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:03.6400378Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:03.6401108Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:03.6402016Z FILE_SUFFIX: test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067 2024-08-20T23:48:03.6402693Z ##[endgroup] 2024-08-20T23:48:03.6499801Z adding: usage_log.txt (deflated 92%) 2024-08-20T23:48:03.6595795Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T23:48:03.6596536Z # Remove any previous debugging artifacts if they exist 2024-08-20T23:48:03.6597098Z rm -f debug-*.zip 2024-08-20T23:48:03.6597488Z if [ -d 'test/debug' ]; then 2024-08-20T23:48:03.6598008Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T23:48:03.6598500Z fi 2024-08-20T23:48:03.6606579Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:48:03.6607089Z env: 2024-08-20T23:48:03.6607372Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:03.6607828Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:03.6608562Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:03.6609484Z FILE_SUFFIX: test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067 2024-08-20T23:48:03.6610163Z ##[endgroup] 2024-08-20T23:48:03.6708263Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:48:03.6708738Z with: 2024-08-20T23:48:03.6709026Z s3-bucket: gha-artifacts 2024-08-20T23:48:03.6709468Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:48:03.6709946Z retention-days: 14 2024-08-20T23:48:03.6710281Z if-no-files-found: warn 2024-08-20T23:48:03.6710639Z path: test-jsons-*.zip 2024-08-20T23:48:03.6711008Z name: artifact 2024-08-20T23:48:03.6711306Z region: us-east-1 2024-08-20T23:48:03.6711610Z env: 2024-08-20T23:48:03.6711889Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:03.6712349Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:03.6713086Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:03.6713725Z ##[endgroup] 2024-08-20T23:48:04.0248649Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:48:04.0249303Z With the provided path, there will be 1 file uploaded 2024-08-20T23:48:04.0249950Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:48:04.1512668Z Starting upload of test-jsons-test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067.zip 2024-08-20T23:48:04.2712380Z Finished upload of test-jsons-test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067.zip 2024-08-20T23:48:04.2997273Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:48:04.2997915Z with: 2024-08-20T23:48:04.2998205Z s3-bucket: gha-artifacts 2024-08-20T23:48:04.2998696Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:48:04.2999174Z retention-days: 14 2024-08-20T23:48:04.2999517Z if-no-files-found: error 2024-08-20T23:48:04.2999885Z path: test-reports-*.zip 2024-08-20T23:48:04.3000241Z name: artifact 2024-08-20T23:48:04.3000542Z region: us-east-1 2024-08-20T23:48:04.3000847Z env: 2024-08-20T23:48:04.3001120Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:04.3001761Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:04.3002490Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:04.3003120Z ##[endgroup] 2024-08-20T23:48:04.6539495Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:48:04.6540118Z With the provided path, there will be 1 file uploaded 2024-08-20T23:48:04.6540741Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:48:04.6597658Z Starting upload of test-reports-test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067.zip 2024-08-20T23:48:04.7511230Z Finished upload of test-reports-test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067.zip 2024-08-20T23:48:04.7794606Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:48:04.7795055Z with: 2024-08-20T23:48:04.7795343Z s3-bucket: gha-artifacts 2024-08-20T23:48:04.7795787Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:48:04.7796280Z retention-days: 14 2024-08-20T23:48:04.7796626Z if-no-files-found: ignore 2024-08-20T23:48:04.7796992Z path: logs-*.zip 2024-08-20T23:48:04.7797304Z name: artifact 2024-08-20T23:48:04.7797606Z region: us-east-1 2024-08-20T23:48:04.7797916Z env: 2024-08-20T23:48:04.7798199Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:04.7798662Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:04.7799390Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:04.7800038Z ##[endgroup] 2024-08-20T23:48:05.1327958Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:48:05.1329201Z With the provided path, there will be 1 file uploaded 2024-08-20T23:48:05.1330769Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:48:05.1385929Z Starting upload of logs-test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067.zip 2024-08-20T23:48:05.2230832Z Finished upload of logs-test-dynamo_eager_torchbench-2-2-linux.g5.4xlarge.nvidia.gpu_29026298067.zip 2024-08-20T23:48:05.2515163Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:48:05.2515619Z with: 2024-08-20T23:48:05.2515903Z s3-bucket: gha-artifacts 2024-08-20T23:48:05.2516348Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:48:05.2516830Z retention-days: 14 2024-08-20T23:48:05.2517177Z if-no-files-found: ignore 2024-08-20T23:48:05.2517625Z path: debug-*.zip 2024-08-20T23:48:05.2518033Z name: artifact 2024-08-20T23:48:05.2518433Z region: us-east-1 2024-08-20T23:48:05.2518841Z env: 2024-08-20T23:48:05.2519186Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:05.2519689Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:05.2520427Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:05.2521061Z ##[endgroup] 2024-08-20T23:48:05.5974910Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T23:48:05.6253404Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T23:48:05.6253893Z # shellcheck disable=SC2156 2024-08-20T23:48:05.6254699Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T23:48:05.6264003Z shell: /usr/bin/bash -e {0} 2024-08-20T23:48:05.6264349Z env: 2024-08-20T23:48:05.6264628Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:05.6265091Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:05.6265998Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:05.6266620Z ##[endgroup] 2024-08-20T23:48:05.8951333Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T23:48:05.8951899Z with: 2024-08-20T23:48:05.8952148Z env: 2024-08-20T23:48:05.8952426Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:05.8952889Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:05.8953623Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:05.8954261Z ##[endgroup] 2024-08-20T23:48:05.8974242Z ##[group]Run set -eou pipefail 2024-08-20T23:48:05.8974659Z set -eou pipefail 2024-08-20T23:48:05.8975026Z  2024-08-20T23:48:05.8975554Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T23:48:05.8976214Z for _ in $(seq 1440); do 2024-08-20T23:48:05.8976694Z  # Break if no ssh session exists anymore 2024-08-20T23:48:05.8977219Z  if [ "$(who)" = "" ]; then 2024-08-20T23:48:05.8977637Z  break 2024-08-20T23:48:05.8977982Z  fi 2024-08-20T23:48:05.8978283Z  echo "." 2024-08-20T23:48:05.8978616Z  sleep 5 2024-08-20T23:48:05.8978941Z done 2024-08-20T23:48:05.8987679Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:48:05.8988186Z env: 2024-08-20T23:48:05.8988478Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:05.8988953Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:05.8989700Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:05.8990339Z ##[endgroup] 2024-08-20T23:48:05.9020657Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T23:48:05.9105729Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:48:05.9106517Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:48:05.9107133Z # shellcheck disable=SC2046 2024-08-20T23:48:05.9107629Z docker stop $(docker ps -q) || true 2024-08-20T23:48:05.9108125Z # Prune all of the docker images 2024-08-20T23:48:05.9108591Z docker system prune -af 2024-08-20T23:48:05.9117355Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:48:05.9117872Z env: 2024-08-20T23:48:05.9118163Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:48:05.9118630Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:48:05.9119363Z DOCKER_CONTAINER_ID: 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:05.9119997Z ##[endgroup] 2024-08-20T23:48:06.7956066Z 1409a3b58649 2024-08-20T23:48:10.8426997Z Deleted Containers: 2024-08-20T23:48:10.8427710Z 1409a3b5864953a659413436036939844b983da7e50ae95e4ae5dc68d7ad602b 2024-08-20T23:48:10.8428297Z 2024-08-20T23:48:18.5515210Z Deleted Images: 2024-08-20T23:48:18.5517224Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T23:48:18.5520027Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T23:48:18.5521596Z deleted: sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb 2024-08-20T23:48:18.5522500Z deleted: sha256:c2ce10014dd98f6abbfbf2265f9780a8204c2a7a27463e61ee565573a23d141d 2024-08-20T23:48:18.5523389Z deleted: sha256:ae249c0e6e6013bd6ffea2ca0b0160c015c9ebf7c03070bc2f983a9ff2415bf5 2024-08-20T23:48:18.5524303Z deleted: sha256:8a73883c1067c0e29ff128f553518682b47ed9707d2f35e7d5138e2caf029d61 2024-08-20T23:48:18.5525142Z deleted: sha256:9c040db729fa084584aaf5573082408a2196bc64e5952270c79dc610ed4680b6 2024-08-20T23:48:18.5526012Z deleted: sha256:17311eeeb8b71b16699880c0f646e185107496c4d41da5d23370603768ec030c 2024-08-20T23:48:18.5527177Z deleted: sha256:07cc10bece8501b1c8035d4385ad9e9867dd6e92fa32218a4dcb3e42af7e0411 2024-08-20T23:48:18.5528237Z deleted: sha256:f3e32d05099d1a3115a4722bc2e94a48752c3e965446773673808252671bb967 2024-08-20T23:48:18.5529080Z deleted: sha256:be7c3f41475e3e72ffe3ac2f92ab2f4cf49d5496b2aae738942ac3bb687d742c 2024-08-20T23:48:18.5529963Z deleted: sha256:4fcac490be825129b468cf458089f1fcf8fd56afc44ef4ab27c2eff41ab1ac90 2024-08-20T23:48:18.5530958Z deleted: sha256:31c66d80a2a4792897f267d363c0cf84fb821c5d4a275ab1636f778fd0284c08 2024-08-20T23:48:18.5531851Z deleted: sha256:8a8bcb6085e7e438e5533e8a5613e2ae5888543853568d023af3e2623a1593a9 2024-08-20T23:48:18.5532838Z deleted: sha256:2f6cb60029c79d11567b738f5bdbfb9cb1838cc0f194c4a5e6ae685178d2ed34 2024-08-20T23:48:18.5533844Z deleted: sha256:e546391804d3313f558e1ac16e93cb29a417a92e4eccaf0d4bd12af3bbcf39e3 2024-08-20T23:48:18.5534691Z deleted: sha256:1623b2aa16f2bdc6b572f6799498d9a92fa8114d07c440599b17aea72e22f8b1 2024-08-20T23:48:18.5535560Z deleted: sha256:c0386ca58b7cb7afee015b7e58d2f70525eaf73d6da2cc63868b20e14b1f9497 2024-08-20T23:48:18.5536425Z deleted: sha256:dcbf534ff289186e246531925fed90f72516eb3d63663bad234d611cdc025d62 2024-08-20T23:48:18.5537268Z deleted: sha256:0bab537af579f56c9cd51c7073fd53635fffb424b363236e0de38a7dd0e60ae8 2024-08-20T23:48:18.5538112Z deleted: sha256:d85a852ae1ee027e2fc17240ed68b87957b0a6d210c9119752d8979a0dcd55cf 2024-08-20T23:48:18.5538948Z deleted: sha256:f3821d761b555362e44cf9a1447d8fd2d95b37d54aea67c7519cd9e5db0c4d7e 2024-08-20T23:48:18.5539793Z deleted: sha256:c05e84a545db5dfcb0847c73d2b0f4749fa67aabce05705632ff7f76c84d4688 2024-08-20T23:48:18.5540643Z deleted: sha256:27054eae7ca67b2bd5ca0fd3f4488bf5c5f78e95487d8c45616f3d753d48a8b5 2024-08-20T23:48:18.5541632Z deleted: sha256:9804139f6ae97796aaa993fe4b89ea0cf111eb4d30a79f4c87c339c2e1e04fa0 2024-08-20T23:48:18.5542478Z deleted: sha256:ad6ae2fc594aaa359ff4e287323836570454eb24dd162930655e414d1d77c6dd 2024-08-20T23:48:18.5543325Z deleted: sha256:9cfbb36e328af52a248853a46aab88e266c09e2b44a03e0a42194a031b9b9453 2024-08-20T23:48:18.5544181Z deleted: sha256:a8a842bd7f37b8d50b6d12d6f95f46bbba57953db466fcbde9ba522625f024ef 2024-08-20T23:48:18.5545038Z deleted: sha256:da9738654b3c86f89ebeed2560f1bae093f1219f1eff90b0394b7f211ecabd04 2024-08-20T23:48:18.5545889Z deleted: sha256:cc683bf0cb245394e0f568984b2dd4fce1b6a3a7fe7c44734bae1aa1caaa3251 2024-08-20T23:48:18.5546740Z deleted: sha256:965f220e71c7d1ec19aaf41b1529ab287beb9d86eb8482c4c14094485ab34475 2024-08-20T23:48:18.5547881Z deleted: sha256:da32ecec41a73ce7dd29464f90c7951510a18cd378753659a78cd7fabd10dc36 2024-08-20T23:48:18.5548716Z deleted: sha256:d828d187cee9ad48dbc3702fb61d359619dd8a1ed582878680b51686bf117e5a 2024-08-20T23:48:18.5549551Z deleted: sha256:712111f4b12ff8e2787597d5fdcb043bd0824e55ddd97f262ef550506ca195f0 2024-08-20T23:48:18.5550378Z deleted: sha256:a064eea405958ad14dc6c794e5df9fd1d4515424272801ec559d45a59317309a 2024-08-20T23:48:18.5551210Z deleted: sha256:e124382d136bd5e9a8b409ca79c1f1ef76458deaf444b1b2060d3fa99dbeb506 2024-08-20T23:48:18.5552048Z deleted: sha256:6e10fe757cc190a1b31b47010f4f7610e02cb1b4b438f190dcd301aa4fe2278e 2024-08-20T23:48:18.5552938Z deleted: sha256:4cea352e10f46ca4729f96ac6c4600fc53d4c36e7658c2e0f19f4f309cd4a2a7 2024-08-20T23:48:18.5553772Z deleted: sha256:518bba00fe1d0933053066d21cac0235c9fb485abbc41557ec184b293095c924 2024-08-20T23:48:18.5554607Z deleted: sha256:a64f9ca42e9f27ff2690c15a5c84312e6efaccad45668d9b496cb542165cfb5a 2024-08-20T23:48:18.5555441Z deleted: sha256:096ea0fa14a6c420658f5276b8a8f5d264c0de7e18161ceda2cfc2f4a92cd9bf 2024-08-20T23:48:18.5556280Z deleted: sha256:6f91c94ce6e3f8034c91c66ee25c66a3d85dba9d813b749bd6d2000942a99edc 2024-08-20T23:48:18.5557121Z deleted: sha256:8f8d255eeae546cead8f9a94d237fa57a99940c0f5d51bc29a9d41c7df9cffd9 2024-08-20T23:48:18.5557975Z deleted: sha256:fe05276f4a49f91eef56bfbc3da87cfff5eb116a827fa2d30efe42ce1c134b40 2024-08-20T23:48:18.5558830Z deleted: sha256:5f83cd75d0669c813d4f58978c92e14305cd8f6a07eb9a60665a583460fa7b59 2024-08-20T23:48:18.5560494Z deleted: sha256:8f7ab6366169d39d8953129f650f1927d2fb1328b3f11e2fd9076adb5f46b93a 2024-08-20T23:48:18.5561670Z deleted: sha256:4146cfd5dba0f5308954749f98596b14a1d8f6d23bd93f7ad86c6c3d72239328 2024-08-20T23:48:18.5562537Z deleted: sha256:c6d83cb9ccc606ff63bb3c4d5a5e1ff1450c619b5aae494303a94ae25971cca0 2024-08-20T23:48:18.5563390Z deleted: sha256:71e037e0a3e0963c4565534a7387c082afcb04475d3ac629acf7a0ab2f49fadc 2024-08-20T23:48:18.5564214Z deleted: sha256:4b31135c564686bdaf23262b901237880aea635655721cc7e8c95dc42b05e06d 2024-08-20T23:48:18.5565049Z deleted: sha256:d09bd5c2f35cedcb846543941ee64ce1f684b52e4285696c93b3fb2032da8579 2024-08-20T23:48:18.5565910Z deleted: sha256:4de7fb7d4fc22212aae5fc1ceb637526916d0cced240d7c93d5edd6b0ea28013 2024-08-20T23:48:18.5566760Z deleted: sha256:69890e3e82746e10aff2f1084f11afb4b95a4a6c26b30f64b7001bc0a5fe32bf 2024-08-20T23:48:18.5567586Z deleted: sha256:0156df53b3224e645336ea52b4bb816f1e8d8c9b591748cd645e5e5649fa507b 2024-08-20T23:48:18.5568423Z deleted: sha256:0664624cb9cc4a386f31bf49919a8d8a11d9fea428d05585d9e6228f754bc7cd 2024-08-20T23:48:18.5569275Z deleted: sha256:8c2ddd5fa535c768c058a23f5cc4b6c64cdd71bb7458491282aabc7f7bb94d69 2024-08-20T23:48:18.5570133Z deleted: sha256:8dd03dbe0e1c5f47dd333074dff4bd00415f2f63727a65bf17866774e2ebbd0d 2024-08-20T23:48:18.5570969Z deleted: sha256:3c56b63e094bd18fad9da7a541d537210bca3b730c590d3641213259f6908d6f 2024-08-20T23:48:18.5571811Z deleted: sha256:51484fef07bd1266d4d6ff3d767751172660fc4cc6a1ffe883523a63c985e31e 2024-08-20T23:48:18.5572648Z deleted: sha256:03e83c94023982ab7dd8dc65a6df2024e342f1b760c91ec7a680ee45824869c7 2024-08-20T23:48:18.5573480Z deleted: sha256:58440ab3e8175b5de94a76485c8689a6dc186015475ebf4b386ebba309c5da34 2024-08-20T23:48:18.5574311Z deleted: sha256:9a980f3383e8aa511f0ef68982cb6dfc53b38d1ac8c3d673cb3ba3d45158c91e 2024-08-20T23:48:18.5575166Z deleted: sha256:cb0269c9cf1c47b6c4188ca7be4db261b5d3fc4b4a5153629e222dba7b1f9f98 2024-08-20T23:48:18.5576039Z deleted: sha256:41f1ba870cdffc05e0bf8d87b640fb9d4c153d7f4f3b775f37f94e9f3d584aad 2024-08-20T23:48:18.5576900Z deleted: sha256:7b502ab352f48e8b6eb5e6824a29f308952a5ee7c7d987562e1df7e47639d7f1 2024-08-20T23:48:18.5577730Z deleted: sha256:6ba5217e18738660e6ca8b7a7ec88845e70b0fd781301520bf46c71d99df3144 2024-08-20T23:48:18.5578565Z deleted: sha256:dd5a1c233594780163726bdbbf5de00f7b6b8dca1dace8a001340714c54baedb 2024-08-20T23:48:18.5579413Z deleted: sha256:c27f0db27da73e01f67f6cb78d10f355e350d829479ba4351d82055bb29f6597 2024-08-20T23:48:18.5580258Z deleted: sha256:4e3ce3c854f35c08be3dd95e0d6e718512a885e4614291d7d8b7cd23f48b7dd8 2024-08-20T23:48:18.5581102Z deleted: sha256:c2d6fb088bbec16e60cf0ad74b016f7287c92e659ec2db01271bc309b4dc6d83 2024-08-20T23:48:18.5582083Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-08-20T23:48:18.5582966Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-08-20T23:48:18.5583783Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-08-20T23:48:18.5584615Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-08-20T23:48:18.5585448Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-08-20T23:48:18.5586278Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-08-20T23:48:18.5587122Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-08-20T23:48:18.5587973Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-08-20T23:48:18.5588817Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-08-20T23:48:18.5589663Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-08-20T23:48:18.5590491Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-20T23:48:18.5590985Z 2024-08-20T23:48:18.5591145Z Total reclaimed space: 49.95GB 2024-08-20T23:48:18.5658867Z Post job cleanup. 2024-08-20T23:48:18.5710467Z Post job cleanup. 2024-08-20T23:48:18.6609525Z [command]/usr/bin/git version 2024-08-20T23:48:18.6659425Z git version 2.40.1 2024-08-20T23:48:18.6690462Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/debb5a78-3bb1-42a7-96c7-01c8029fb524/.gitconfig' 2024-08-20T23:48:18.6701255Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/debb5a78-3bb1-42a7-96c7-01c8029fb524' before making global git config changes 2024-08-20T23:48:18.6702597Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T23:48:18.6705958Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T23:48:18.6760020Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T23:48:18.6804856Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T23:48:18.7184763Z Entering 'android/libs/fbjni' 2024-08-20T23:48:18.7256565Z Entering 'third_party/FP16' 2024-08-20T23:48:18.7327803Z Entering 'third_party/FXdiv' 2024-08-20T23:48:18.7399284Z Entering 'third_party/NNPACK' 2024-08-20T23:48:18.7468675Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:48:18.7537027Z Entering 'third_party/XNNPACK' 2024-08-20T23:48:18.7624520Z Entering 'third_party/benchmark' 2024-08-20T23:48:18.7694299Z Entering 'third_party/cpp-httplib' 2024-08-20T23:48:18.7765177Z Entering 'third_party/cpuinfo' 2024-08-20T23:48:18.7834764Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:48:18.7905490Z Entering 'third_party/cutlass' 2024-08-20T23:48:18.7984256Z Entering 'third_party/eigen' 2024-08-20T23:48:18.8059393Z Entering 'third_party/fbgemm' 2024-08-20T23:48:18.8126804Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:48:18.8194467Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:48:18.8266035Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:48:18.8340388Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:48:18.8408550Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:48:18.8479655Z Entering 'third_party/flatbuffers' 2024-08-20T23:48:18.8554162Z Entering 'third_party/fmt' 2024-08-20T23:48:18.8624723Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:48:18.8693880Z Entering 'third_party/gloo' 2024-08-20T23:48:18.8767462Z Entering 'third_party/googletest' 2024-08-20T23:48:18.8836753Z Entering 'third_party/ideep' 2024-08-20T23:48:18.8903065Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:48:18.8979653Z Entering 'third_party/ittapi' 2024-08-20T23:48:18.9050463Z Entering 'third_party/kineto' 2024-08-20T23:48:18.9118552Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:48:18.9185199Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:48:18.9255912Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:48:18.9324832Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:48:18.9393087Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:48:18.9459060Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:48:18.9530997Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:48:18.9604979Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:48:18.9675130Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:48:18.9744454Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:48:18.9820664Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:48:18.9891336Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:48:18.9962897Z Entering 'third_party/mimalloc' 2024-08-20T23:48:19.0031430Z Entering 'third_party/nccl/nccl' 2024-08-20T23:48:19.0101901Z Entering 'third_party/nlohmann' 2024-08-20T23:48:19.0171967Z Entering 'third_party/onnx' 2024-08-20T23:48:19.0255380Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:48:19.0323319Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:48:19.0399689Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:48:19.0472383Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:48:19.0538585Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:48:19.0606608Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:48:19.0673170Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:48:19.0742036Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:48:19.0808987Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:48:19.0877351Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:48:19.0943590Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:48:19.1012329Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:48:19.1089263Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:48:19.1179913Z Entering 'third_party/pocketfft' 2024-08-20T23:48:19.1249048Z Entering 'third_party/protobuf' 2024-08-20T23:48:19.1319348Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:48:19.1387263Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:48:19.1459320Z Entering 'third_party/psimd' 2024-08-20T23:48:19.1527342Z Entering 'third_party/pthreadpool' 2024-08-20T23:48:19.1598284Z Entering 'third_party/pybind11' 2024-08-20T23:48:19.1668058Z Entering 'third_party/python-peachpy' 2024-08-20T23:48:19.1737010Z Entering 'third_party/sleef' 2024-08-20T23:48:19.1806628Z Entering 'third_party/tensorpipe' 2024-08-20T23:48:19.1877477Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:48:19.1945038Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:48:19.2012073Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:48:19.2079689Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:48:19.2144262Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:48:19.2240373Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T23:48:19.2277758Z http.https://github.com/.extraheader 2024-08-20T23:48:19.2287672Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T23:48:19.2334463Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T23:48:19.2700000Z Entering 'android/libs/fbjni' 2024-08-20T23:48:19.2743415Z http.https://github.com/.extraheader 2024-08-20T23:48:19.2788960Z Entering 'third_party/FP16' 2024-08-20T23:48:19.2832821Z http.https://github.com/.extraheader 2024-08-20T23:48:19.2878405Z Entering 'third_party/FXdiv' 2024-08-20T23:48:19.2923511Z http.https://github.com/.extraheader 2024-08-20T23:48:19.2968573Z Entering 'third_party/NNPACK' 2024-08-20T23:48:19.3013316Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3058596Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:48:19.3103294Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3148770Z Entering 'third_party/XNNPACK' 2024-08-20T23:48:19.3193235Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3254707Z Entering 'third_party/benchmark' 2024-08-20T23:48:19.3299063Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3343568Z Entering 'third_party/cpp-httplib' 2024-08-20T23:48:19.3389079Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3433034Z Entering 'third_party/cpuinfo' 2024-08-20T23:48:19.3477764Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3523079Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:48:19.3567619Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3612006Z Entering 'third_party/cutlass' 2024-08-20T23:48:19.3657190Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3709239Z Entering 'third_party/eigen' 2024-08-20T23:48:19.3755283Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3803252Z Entering 'third_party/fbgemm' 2024-08-20T23:48:19.3848147Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3892540Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:48:19.3935028Z http.https://github.com/.extraheader 2024-08-20T23:48:19.3978414Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:48:19.4021261Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4064760Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:48:19.4112035Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4161390Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:48:19.4205403Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4248192Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:48:19.4292953Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4338454Z Entering 'third_party/flatbuffers' 2024-08-20T23:48:19.4382226Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4427064Z Entering 'third_party/fmt' 2024-08-20T23:48:19.4471426Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4513769Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:48:19.4557512Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4599829Z Entering 'third_party/gloo' 2024-08-20T23:48:19.4643511Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4691919Z Entering 'third_party/googletest' 2024-08-20T23:48:19.4735534Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4779042Z Entering 'third_party/ideep' 2024-08-20T23:48:19.4822672Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4866812Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:48:19.4909106Z http.https://github.com/.extraheader 2024-08-20T23:48:19.4960660Z Entering 'third_party/ittapi' 2024-08-20T23:48:19.5004980Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5047336Z Entering 'third_party/kineto' 2024-08-20T23:48:19.5090605Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5132656Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:48:19.5177062Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5220207Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:48:19.5264035Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5308734Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:48:19.5355754Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5400968Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:48:19.5442492Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5487906Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:48:19.5529800Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5572803Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:48:19.5615072Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5662918Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:48:19.5704742Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5748735Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:48:19.5790455Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5834578Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:48:19.5882838Z http.https://github.com/.extraheader 2024-08-20T23:48:19.5931233Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:48:19.5976968Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6024635Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:48:19.6068058Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6111256Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:48:19.6153893Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6200795Z Entering 'third_party/mimalloc' 2024-08-20T23:48:19.6246352Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6293166Z Entering 'third_party/nccl/nccl' 2024-08-20T23:48:19.6337215Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6382626Z Entering 'third_party/nlohmann' 2024-08-20T23:48:19.6426488Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6472289Z Entering 'third_party/onnx' 2024-08-20T23:48:19.6516475Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6575250Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:48:19.6624089Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6670712Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:48:19.6713806Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6764403Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:48:19.6808792Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6855861Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:48:19.6902839Z http.https://github.com/.extraheader 2024-08-20T23:48:19.6947773Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:48:19.6989984Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7033846Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:48:19.7076553Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7121288Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:48:19.7164548Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7209998Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:48:19.7253025Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7295970Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:48:19.7338295Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7381051Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:48:19.7423329Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7465553Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:48:19.7508196Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7554793Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:48:19.7597884Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7645948Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:48:19.7692653Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7759329Z Entering 'third_party/pocketfft' 2024-08-20T23:48:19.7803960Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7850892Z Entering 'third_party/protobuf' 2024-08-20T23:48:19.7895509Z http.https://github.com/.extraheader 2024-08-20T23:48:19.7943138Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:48:19.7990810Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8033588Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:48:19.8077331Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8126141Z Entering 'third_party/psimd' 2024-08-20T23:48:19.8169484Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8214041Z Entering 'third_party/pthreadpool' 2024-08-20T23:48:19.8257346Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8301104Z Entering 'third_party/pybind11' 2024-08-20T23:48:19.8344887Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8389804Z Entering 'third_party/python-peachpy' 2024-08-20T23:48:19.8433757Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8479012Z Entering 'third_party/sleef' 2024-08-20T23:48:19.8522951Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8569888Z Entering 'third_party/tensorpipe' 2024-08-20T23:48:19.8614411Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8658863Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:48:19.8701507Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8746288Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:48:19.8789251Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8832071Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:48:19.8874662Z http.https://github.com/.extraheader 2024-08-20T23:48:19.8917389Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:48:19.8961626Z http.https://github.com/.extraheader 2024-08-20T23:48:19.9004579Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:48:19.9047376Z http.https://github.com/.extraheader 2024-08-20T23:48:19.9213049Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T23:48:19.9233749Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T23:48:19.9241820Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:48:19.9242352Z ##[endgroup] 2024-08-20T23:48:28.0157301Z Cleaning up orphan processes