2024-08-20T22:04:31.7604312Z Current runner version: '2.319.1' 2024-08-20T22:04:31.7610588Z Runner name: 'i-02d1d9ee19baacafb' 2024-08-20T22:04:31.7611402Z Runner group name: 'Default' 2024-08-20T22:04:31.7612367Z Machine name: 'ip-10-0-46-136' 2024-08-20T22:04:31.7629096Z Testing runner upgrade compatibility 2024-08-20T22:04:31.9138706Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T22:04:31.9141066Z Actions: read 2024-08-20T22:04:31.9141558Z Attestations: read 2024-08-20T22:04:31.9142187Z Checks: read 2024-08-20T22:04:31.9142677Z Contents: read 2024-08-20T22:04:31.9143183Z Deployments: read 2024-08-20T22:04:31.9143749Z Discussions: read 2024-08-20T22:04:31.9144264Z Issues: read 2024-08-20T22:04:31.9144769Z Metadata: read 2024-08-20T22:04:31.9145282Z Packages: read 2024-08-20T22:04:31.9145785Z Pages: read 2024-08-20T22:04:31.9146291Z PullRequests: read 2024-08-20T22:04:31.9146893Z RepositoryProjects: read 2024-08-20T22:04:31.9147456Z SecurityEvents: read 2024-08-20T22:04:31.9148014Z Statuses: read 2024-08-20T22:04:31.9148752Z ##[endgroup] 2024-08-20T22:04:31.9151888Z Secret source: Actions 2024-08-20T22:04:31.9152600Z Prepare workflow directory 2024-08-20T22:04:32.0079995Z Prepare all required actions 2024-08-20T22:04:32.0246800Z Getting action download info 2024-08-20T22:04:32.2262082Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T22:04:32.6726197Z Download action repository 'pytorch/pytorch@main' (SHA:1ae5d5bb62141d7c9b1b0b66c66a462b4e10b1f2) 2024-08-20T22:04:36.7052713Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T22:04:36.9002748Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T22:04:37.2262682Z Getting action download info 2024-08-20T22:04:37.3259125Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T22:04:38.5441237Z Getting action download info 2024-08-20T22:04:38.6568449Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T22:04:38.8425422Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T22:04:38.8427599Z ##[group] Inputs 2024-08-20T22:04:38.8428082Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:04:38.8435548Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:04:38.8443463Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:04:38.8444659Z sync-tag: 2024-08-20T22:04:38.8445557Z timeout-minutes: 240 2024-08-20T22:04:38.8445905Z use-gha: 2024-08-20T22:04:38.8446201Z dashboard-tag: 2024-08-20T22:04:38.8446534Z s3-bucket: gha-artifacts 2024-08-20T22:04:38.8446893Z aws-role-to-assume: 2024-08-20T22:04:38.8447237Z ##[endgroup] 2024-08-20T22:04:38.8448254Z Complete job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:04:38.9186425Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T22:04:38.9326722Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T22:04:38.9338426Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:04:38.9338978Z ##[endgroup] 2024-08-20T22:04:40.0727881Z Runner Type: linux.g5.4xlarge.nvidia.gpu 2024-08-20T22:04:40.0728409Z Instance Type: g5.4xlarge 2024-08-20T22:04:40.0729115Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T22:04:40.0729662Z AMI ID: ami-06c68f701d8090592 2024-08-20T22:04:45.8651382Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T22:04:45.8651965Z with: 2024-08-20T22:04:45.8652649Z github-secret: *** 2024-08-20T22:04:45.8653570Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:04:45.8654559Z activate-with-label: false 2024-08-20T22:04:45.8654923Z label: with-ssh 2024-08-20T22:04:45.8655250Z remove-existing-keys: true 2024-08-20T22:04:45.8655609Z fail-silently: true 2024-08-20T22:04:45.8655930Z env: 2024-08-20T22:04:45.8656211Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:04:45.8656573Z ##[endgroup] 2024-08-20T22:04:45.9552927Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T22:04:45.9555653Z ciflow reference detected, attempting to extract PR number 2024-08-20T22:04:46.3005301Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T22:04:46.3728018Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T22:04:46.3728633Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T22:04:46.4714710Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T22:04:46.4730856Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T22:04:46.4755378Z Login using: ssh ec2-user@ec2-34-207-70-182.compute-1.amazonaws.com 2024-08-20T22:04:46.4756149Z All testing is done inside the container, to start an interactive session run: 2024-08-20T22:04:46.4756949Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:04:46.4889473Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T22:04:46.4890036Z with: 2024-08-20T22:04:46.4890328Z submodules: recursive 2024-08-20T22:04:46.4890667Z fetch-depth: 0 2024-08-20T22:04:46.4890967Z env: 2024-08-20T22:04:46.4891249Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:04:46.4891591Z ##[endgroup] 2024-08-20T22:04:46.5092764Z ##[group]Run retry () { 2024-08-20T22:04:46.5093423Z retry () { 2024-08-20T22:04:46.5093949Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T22:04:46.5094541Z } 2024-08-20T22:04:46.5094882Z echo "${GITHUB_WORKSPACE}" 2024-08-20T22:04:46.5095306Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T22:04:46.5095788Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:04:46.5096245Z else 2024-08-20T22:04:46.5096586Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:04:46.5097013Z fi 2024-08-20T22:04:46.5097556Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T22:04:46.5110841Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:04:46.5111359Z env: 2024-08-20T22:04:46.5111645Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:04:46.5111993Z NO_SUDO: 2024-08-20T22:04:46.5112285Z ##[endgroup] 2024-08-20T22:04:46.5144067Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:04:46.6819545Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T22:04:46.6819989Z with: 2024-08-20T22:04:46.6820326Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:04:46.6820774Z fetch-depth: 0 2024-08-20T22:04:46.6821213Z submodules: recursive 2024-08-20T22:04:46.6821557Z quiet-checkout: true 2024-08-20T22:04:46.6821916Z repository: pytorch/pytorch 2024-08-20T22:04:46.6822408Z token: *** 2024-08-20T22:04:46.6822708Z ssh-strict: true 2024-08-20T22:04:46.6823037Z persist-credentials: true 2024-08-20T22:04:46.6823401Z clean: true 2024-08-20T22:04:46.6823745Z sparse-checkout-cone-mode: true 2024-08-20T22:04:46.6824151Z lfs: false 2024-08-20T22:04:46.6824456Z set-safe-directory: true 2024-08-20T22:04:46.6824802Z env: 2024-08-20T22:04:46.6825083Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:04:46.6825429Z ##[endgroup] 2024-08-20T22:04:46.7791387Z Syncing repository: pytorch/pytorch 2024-08-20T22:04:46.7792937Z ##[group]Getting Git version info 2024-08-20T22:04:46.7794085Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:04:46.7794948Z [command]/usr/bin/git version 2024-08-20T22:04:46.7795312Z git version 2.40.1 2024-08-20T22:04:46.7798543Z ##[endgroup] 2024-08-20T22:04:46.7812881Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/0d2f999a-b66b-4ebf-a7a5-4efd322eca9b' before making global git config changes 2024-08-20T22:04:46.7814136Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:04:46.7818021Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:04:46.7867235Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:04:46.7870725Z ##[group]Initializing the repository 2024-08-20T22:04:46.7873588Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:04:46.7918065Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T22:04:46.7919197Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T22:04:46.7920047Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T22:04:46.7920618Z hint: 2024-08-20T22:04:46.7921071Z hint: git config --global init.defaultBranch 2024-08-20T22:04:46.7921683Z hint: 2024-08-20T22:04:46.7922258Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T22:04:46.7923129Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T22:04:46.7924045Z hint: 2024-08-20T22:04:46.7924387Z hint: git branch -m 2024-08-20T22:04:46.7926097Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T22:04:46.7934044Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T22:04:46.7976886Z ##[endgroup] 2024-08-20T22:04:46.7977555Z ##[group]Disabling automatic garbage collection 2024-08-20T22:04:46.7979940Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T22:04:46.8020238Z ##[endgroup] 2024-08-20T22:04:46.8020851Z ##[group]Setting up auth 2024-08-20T22:04:46.8026735Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:04:46.8069789Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:04:46.8447771Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:04:46.8490271Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:04:46.8843769Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:04:46.8901134Z ##[endgroup] 2024-08-20T22:04:46.8901846Z ##[group]Fetching the repository 2024-08-20T22:04:46.8908077Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T22:04:49.3823851Z remote: Enumerating objects: 1008518 2024-08-20T22:04:49.3824410Z remote: Enumerating objects: 1010199, done. 2024-08-20T22:04:49.3825984Z remote: Counting objects: 0% (1/1681) 2024-08-20T22:04:49.3827358Z remote: Counting objects: 1% (17/1681) 2024-08-20T22:04:49.3829309Z remote: Counting objects: 2% (34/1681) 2024-08-20T22:04:49.3830612Z remote: Counting objects: 3% (51/1681) 2024-08-20T22:04:49.3831890Z remote: Counting objects: 4% (68/1681) 2024-08-20T22:04:49.3833199Z remote: Counting objects: 5% (85/1681) 2024-08-20T22:04:49.3834527Z remote: Counting objects: 6% (101/1681) 2024-08-20T22:04:49.3835351Z remote: Counting objects: 7% (118/1681) 2024-08-20T22:04:49.3836002Z remote: Counting objects: 8% (135/1681) 2024-08-20T22:04:49.3836672Z remote: Counting objects: 9% (152/1681) 2024-08-20T22:04:49.3837328Z remote: Counting objects: 10% (169/1681) 2024-08-20T22:04:49.3837985Z remote: Counting objects: 11% (185/1681) 2024-08-20T22:04:49.3838640Z remote: Counting objects: 12% (202/1681) 2024-08-20T22:04:49.3839296Z remote: Counting objects: 13% (219/1681) 2024-08-20T22:04:49.3839835Z remote: Counting objects: 14% (236/1681) 2024-08-20T22:04:49.3840527Z remote: Counting objects: 15% (253/1681) 2024-08-20T22:04:49.3841181Z remote: Counting objects: 16% (269/1681) 2024-08-20T22:04:49.3841713Z remote: Counting objects: 17% (286/1681) 2024-08-20T22:04:49.3842226Z remote: Counting objects: 18% (303/1681) 2024-08-20T22:04:49.3842754Z remote: Counting objects: 19% (320/1681) 2024-08-20T22:04:49.3843272Z remote: Counting objects: 20% (337/1681) 2024-08-20T22:04:49.3843931Z remote: Counting objects: 21% (354/1681) 2024-08-20T22:04:49.3844568Z remote: Counting objects: 22% (370/1681) 2024-08-20T22:04:49.3845213Z remote: Counting objects: 23% (387/1681) 2024-08-20T22:04:49.3845880Z remote: Counting objects: 24% (404/1681) 2024-08-20T22:04:49.3846375Z remote: Counting objects: 25% (421/1681) 2024-08-20T22:04:49.3846866Z remote: Counting objects: 26% (438/1681) 2024-08-20T22:04:49.3847362Z remote: Counting objects: 27% (454/1681) 2024-08-20T22:04:49.3848162Z remote: Counting objects: 28% (471/1681) 2024-08-20T22:04:49.3848670Z remote: Counting objects: 29% (488/1681) 2024-08-20T22:04:49.3849163Z remote: Counting objects: 30% (505/1681) 2024-08-20T22:04:49.3849653Z remote: Counting objects: 31% (522/1681) 2024-08-20T22:04:49.3850142Z remote: Counting objects: 32% (538/1681) 2024-08-20T22:04:49.3850629Z remote: Counting objects: 33% (555/1681) 2024-08-20T22:04:49.3851122Z remote: Counting objects: 34% (572/1681) 2024-08-20T22:04:49.3851616Z remote: Counting objects: 35% (589/1681) 2024-08-20T22:04:49.3852107Z remote: Counting objects: 36% (606/1681) 2024-08-20T22:04:49.3852601Z remote: Counting objects: 37% (622/1681) 2024-08-20T22:04:49.3853094Z remote: Counting objects: 38% (639/1681) 2024-08-20T22:04:49.3853584Z remote: Counting objects: 39% (656/1681) 2024-08-20T22:04:49.3854067Z remote: Counting objects: 40% (673/1681) 2024-08-20T22:04:49.3854726Z remote: Counting objects: 41% (690/1681) 2024-08-20T22:04:49.3855218Z remote: Counting objects: 42% (707/1681) 2024-08-20T22:04:49.3855709Z remote: Counting objects: 43% (723/1681) 2024-08-20T22:04:49.3856193Z remote: Counting objects: 44% (740/1681) 2024-08-20T22:04:49.3856688Z remote: Counting objects: 45% (757/1681) 2024-08-20T22:04:49.3857182Z remote: Counting objects: 46% (774/1681) 2024-08-20T22:04:49.3857675Z remote: Counting objects: 47% (791/1681) 2024-08-20T22:04:49.3858171Z remote: Counting objects: 48% (807/1681) 2024-08-20T22:04:49.3858664Z remote: Counting objects: 49% (824/1681) 2024-08-20T22:04:49.3859157Z remote: Counting objects: 50% (841/1681) 2024-08-20T22:04:49.3859811Z remote: Counting objects: 51% (858/1681) 2024-08-20T22:04:49.3860294Z remote: Counting objects: 52% (875/1681) 2024-08-20T22:04:49.3860797Z remote: Counting objects: 53% (891/1681) 2024-08-20T22:04:49.3861303Z remote: Counting objects: 54% (908/1681) 2024-08-20T22:04:49.3861790Z remote: Counting objects: 55% (925/1681) 2024-08-20T22:04:49.3862270Z remote: Counting objects: 56% (942/1681) 2024-08-20T22:04:49.3862864Z remote: Counting objects: 57% (959/1681) 2024-08-20T22:04:49.3863421Z remote: Counting objects: 58% (975/1681) 2024-08-20T22:04:49.3863919Z remote: Counting objects: 59% (992/1681) 2024-08-20T22:04:49.3864423Z remote: Counting objects: 60% (1009/1681) 2024-08-20T22:04:49.3864940Z remote: Counting objects: 61% (1026/1681) 2024-08-20T22:04:49.3865457Z remote: Counting objects: 62% (1043/1681) 2024-08-20T22:04:49.3865983Z remote: Counting objects: 63% (1060/1681) 2024-08-20T22:04:49.3866500Z remote: Counting objects: 64% (1076/1681) 2024-08-20T22:04:49.3867002Z remote: Counting objects: 65% (1093/1681) 2024-08-20T22:04:49.3867518Z remote: Counting objects: 66% (1110/1681) 2024-08-20T22:04:49.3868037Z remote: Counting objects: 67% (1127/1681) 2024-08-20T22:04:49.3868542Z remote: Counting objects: 68% (1144/1681) 2024-08-20T22:04:49.3869050Z remote: Counting objects: 69% (1160/1681) 2024-08-20T22:04:49.3869555Z remote: Counting objects: 70% (1177/1681) 2024-08-20T22:04:49.3870065Z remote: Counting objects: 71% (1194/1681) 2024-08-20T22:04:49.3870575Z remote: Counting objects: 72% (1211/1681) 2024-08-20T22:04:49.3871084Z remote: Counting objects: 73% (1228/1681) 2024-08-20T22:04:49.3871589Z remote: Counting objects: 74% (1244/1681) 2024-08-20T22:04:49.3872095Z remote: Counting objects: 75% (1261/1681) 2024-08-20T22:04:49.3872598Z remote: Counting objects: 76% (1278/1681) 2024-08-20T22:04:49.3873107Z remote: Counting objects: 77% (1295/1681) 2024-08-20T22:04:49.3873616Z remote: Counting objects: 78% (1312/1681) 2024-08-20T22:04:49.3874243Z remote: Counting objects: 79% (1328/1681) 2024-08-20T22:04:49.3874773Z remote: Counting objects: 80% (1345/1681) 2024-08-20T22:04:49.3875280Z remote: Counting objects: 81% (1362/1681) 2024-08-20T22:04:49.3875791Z remote: Counting objects: 82% (1379/1681) 2024-08-20T22:04:49.3876302Z remote: Counting objects: 83% (1396/1681) 2024-08-20T22:04:49.3876810Z remote: Counting objects: 84% (1413/1681) 2024-08-20T22:04:49.3877315Z remote: Counting objects: 85% (1429/1681) 2024-08-20T22:04:49.3877838Z remote: Counting objects: 86% (1446/1681) 2024-08-20T22:04:49.3878348Z remote: Counting objects: 87% (1463/1681) 2024-08-20T22:04:49.3878857Z remote: Counting objects: 88% (1480/1681) 2024-08-20T22:04:49.3879359Z remote: Counting objects: 89% (1497/1681) 2024-08-20T22:04:49.3879868Z remote: Counting objects: 90% (1513/1681) 2024-08-20T22:04:49.3880376Z remote: Counting objects: 91% (1530/1681) 2024-08-20T22:04:49.3880895Z remote: Counting objects: 92% (1547/1681) 2024-08-20T22:04:49.3881502Z remote: Counting objects: 93% (1564/1681) 2024-08-20T22:04:49.3882010Z remote: Counting objects: 94% (1581/1681) 2024-08-20T22:04:49.3882515Z remote: Counting objects: 95% (1597/1681) 2024-08-20T22:04:49.3883021Z remote: Counting objects: 96% (1614/1681) 2024-08-20T22:04:49.3883523Z remote: Counting objects: 97% (1631/1681) 2024-08-20T22:04:49.3884033Z remote: Counting objects: 98% (1648/1681) 2024-08-20T22:04:49.3884541Z remote: Counting objects: 99% (1665/1681) 2024-08-20T22:04:49.3885045Z remote: Counting objects: 100% (1681/1681) 2024-08-20T22:04:49.3885588Z remote: Counting objects: 100% (1681/1681), done. 2024-08-20T22:04:49.3943328Z remote: Compressing objects: 0% (1/866) 2024-08-20T22:04:49.4254769Z remote: Compressing objects: 1% (9/866) 2024-08-20T22:04:49.4461504Z remote: Compressing objects: 2% (18/866) 2024-08-20T22:04:49.4888904Z remote: Compressing objects: 3% (26/866) 2024-08-20T22:04:49.5700933Z remote: Compressing objects: 4% (35/866) 2024-08-20T22:04:49.6114001Z remote: Compressing objects: 5% (44/866) 2024-08-20T22:04:49.6651577Z remote: Compressing objects: 6% (52/866) 2024-08-20T22:04:49.7081416Z remote: Compressing objects: 7% (61/866) 2024-08-20T22:04:49.7413899Z remote: Compressing objects: 8% (70/866) 2024-08-20T22:04:49.7677778Z remote: Compressing objects: 9% (78/866) 2024-08-20T22:04:49.7910694Z remote: Compressing objects: 10% (87/866) 2024-08-20T22:04:49.8218113Z remote: Compressing objects: 11% (96/866) 2024-08-20T22:04:49.8433628Z remote: Compressing objects: 12% (104/866) 2024-08-20T22:04:49.8626055Z remote: Compressing objects: 13% (113/866) 2024-08-20T22:04:49.8748601Z remote: Compressing objects: 14% (122/866) 2024-08-20T22:04:49.8886752Z remote: Compressing objects: 15% (130/866) 2024-08-20T22:04:49.8966218Z remote: Compressing objects: 16% (139/866) 2024-08-20T22:04:49.9051319Z remote: Compressing objects: 17% (148/866) 2024-08-20T22:04:49.9112707Z remote: Compressing objects: 18% (156/866) 2024-08-20T22:04:49.9166300Z remote: Compressing objects: 19% (165/866) 2024-08-20T22:04:49.9206493Z remote: Compressing objects: 20% (174/866) 2024-08-20T22:04:49.9232217Z remote: Compressing objects: 21% (182/866) 2024-08-20T22:04:49.9250167Z remote: Compressing objects: 22% (191/866) 2024-08-20T22:04:49.9253172Z remote: Compressing objects: 23% (200/866) 2024-08-20T22:04:49.9260891Z remote: Compressing objects: 24% (208/866) 2024-08-20T22:04:49.9270052Z remote: Compressing objects: 25% (217/866) 2024-08-20T22:04:49.9274751Z remote: Compressing objects: 26% (226/866) 2024-08-20T22:04:49.9290599Z remote: Compressing objects: 27% (234/866) 2024-08-20T22:04:49.9313477Z remote: Compressing objects: 28% (243/866) 2024-08-20T22:04:49.9323057Z remote: Compressing objects: 29% (252/866) 2024-08-20T22:04:49.9334939Z remote: Compressing objects: 30% (260/866) 2024-08-20T22:04:49.9343083Z remote: Compressing objects: 31% (269/866) 2024-08-20T22:04:49.9350393Z remote: Compressing objects: 32% (278/866) 2024-08-20T22:04:49.9355229Z remote: Compressing objects: 33% (286/866) 2024-08-20T22:04:49.9364816Z remote: Compressing objects: 34% (295/866) 2024-08-20T22:04:49.9370110Z remote: Compressing objects: 35% (304/866) 2024-08-20T22:04:49.9379797Z remote: Compressing objects: 36% (312/866) 2024-08-20T22:04:49.9386161Z remote: Compressing objects: 37% (321/866) 2024-08-20T22:04:49.9391606Z remote: Compressing objects: 38% (330/866) 2024-08-20T22:04:49.9398485Z remote: Compressing objects: 39% (338/866) 2024-08-20T22:04:49.9403762Z remote: Compressing objects: 40% (347/866) 2024-08-20T22:04:49.9411851Z remote: Compressing objects: 41% (356/866) 2024-08-20T22:04:49.9414857Z remote: Compressing objects: 42% (364/866) 2024-08-20T22:04:49.9422651Z remote: Compressing objects: 43% (373/866) 2024-08-20T22:04:49.9426980Z remote: Compressing objects: 44% (382/866) 2024-08-20T22:04:49.9433008Z remote: Compressing objects: 45% (390/866) 2024-08-20T22:04:49.9436534Z remote: Compressing objects: 46% (399/866) 2024-08-20T22:04:49.9439588Z remote: Compressing objects: 47% (408/866) 2024-08-20T22:04:49.9443494Z remote: Compressing objects: 48% (416/866) 2024-08-20T22:04:49.9446642Z remote: Compressing objects: 49% (425/866) 2024-08-20T22:04:49.9450150Z remote: Compressing objects: 50% (433/866) 2024-08-20T22:04:49.9453265Z remote: Compressing objects: 51% (442/866) 2024-08-20T22:04:49.9453787Z remote: Compressing objects: 52% (451/866) 2024-08-20T22:04:49.9455922Z remote: Compressing objects: 53% (459/866) 2024-08-20T22:04:49.9457714Z remote: Compressing objects: 54% (468/866) 2024-08-20T22:04:49.9458871Z remote: Compressing objects: 55% (477/866) 2024-08-20T22:04:49.9462960Z remote: Compressing objects: 56% (485/866) 2024-08-20T22:04:49.9463485Z remote: Compressing objects: 57% (494/866) 2024-08-20T22:04:49.9464010Z remote: Compressing objects: 58% (503/866) 2024-08-20T22:04:49.9464529Z remote: Compressing objects: 59% (511/866) 2024-08-20T22:04:49.9465061Z remote: Compressing objects: 60% (520/866) 2024-08-20T22:04:49.9465580Z remote: Compressing objects: 61% (529/866) 2024-08-20T22:04:49.9466101Z remote: Compressing objects: 62% (537/866) 2024-08-20T22:04:49.9466621Z remote: Compressing objects: 63% (546/866) 2024-08-20T22:04:49.9472208Z remote: Compressing objects: 64% (555/866) 2024-08-20T22:04:49.9478541Z remote: Compressing objects: 65% (563/866) 2024-08-20T22:04:49.9482764Z remote: Compressing objects: 66% (572/866) 2024-08-20T22:04:49.9487196Z remote: Compressing objects: 67% (581/866) 2024-08-20T22:04:49.9491849Z remote: Compressing objects: 68% (589/866) 2024-08-20T22:04:49.9496357Z remote: Compressing objects: 69% (598/866) 2024-08-20T22:04:49.9500334Z remote: Compressing objects: 70% (607/866) 2024-08-20T22:04:49.9503784Z remote: Compressing objects: 71% (615/866) 2024-08-20T22:04:49.9507189Z remote: Compressing objects: 72% (624/866) 2024-08-20T22:04:49.9510994Z remote: Compressing objects: 73% (633/866) 2024-08-20T22:04:49.9514185Z remote: Compressing objects: 74% (641/866) 2024-08-20T22:04:49.9516194Z remote: Compressing objects: 75% (650/866) 2024-08-20T22:04:49.9518835Z remote: Compressing objects: 76% (659/866) 2024-08-20T22:04:49.9521987Z remote: Compressing objects: 77% (667/866) 2024-08-20T22:04:49.9523603Z remote: Compressing objects: 78% (676/866) 2024-08-20T22:04:49.9526452Z remote: Compressing objects: 79% (685/866) 2024-08-20T22:04:49.9527612Z remote: Compressing objects: 80% (693/866) 2024-08-20T22:04:49.9529940Z remote: Compressing objects: 81% (702/866) 2024-08-20T22:04:49.9532578Z remote: Compressing objects: 82% (711/866) 2024-08-20T22:04:49.9533129Z remote: Compressing objects: 83% (719/866) 2024-08-20T22:04:49.9535874Z remote: Compressing objects: 84% (728/866) 2024-08-20T22:04:49.9539158Z remote: Compressing objects: 85% (737/866) 2024-08-20T22:04:49.9540613Z remote: Compressing objects: 86% (745/866) 2024-08-20T22:04:49.9541156Z remote: Compressing objects: 87% (754/866) 2024-08-20T22:04:49.9543012Z remote: Compressing objects: 88% (763/866) 2024-08-20T22:04:49.9545439Z remote: Compressing objects: 89% (771/866) 2024-08-20T22:04:49.9546663Z remote: Compressing objects: 90% (780/866) 2024-08-20T22:04:49.9547465Z remote: Compressing objects: 91% (789/866) 2024-08-20T22:04:49.9548504Z remote: Compressing objects: 92% (797/866) 2024-08-20T22:04:49.9549063Z remote: Compressing objects: 93% (806/866) 2024-08-20T22:04:49.9549606Z remote: Compressing objects: 94% (815/866) 2024-08-20T22:04:49.9550882Z remote: Compressing objects: 95% (823/866) 2024-08-20T22:04:49.9552815Z remote: Compressing objects: 96% (832/866) 2024-08-20T22:04:49.9553357Z remote: Compressing objects: 97% (841/866) 2024-08-20T22:04:49.9556053Z remote: Compressing objects: 98% (849/866) 2024-08-20T22:04:49.9556588Z remote: Compressing objects: 99% (858/866) 2024-08-20T22:04:49.9557124Z remote: Compressing objects: 100% (866/866) 2024-08-20T22:04:49.9557698Z remote: Compressing objects: 100% (866/866), done. 2024-08-20T22:05:10.4341934Z remote: Total 1010199 (delta 1081), reused 1297 (delta 812), pack-reused 1008518 (from 1) 2024-08-20T22:05:35.5883170Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T22:05:35.5918554Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:05:35.5924283Z ##[endgroup] 2024-08-20T22:05:35.5929875Z ##[group]Determining the checkout info 2024-08-20T22:05:35.5930601Z ##[endgroup] 2024-08-20T22:05:35.5931138Z ##[group]Checking out the ref 2024-08-20T22:05:35.5932079Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:05:37.3215977Z ##[endgroup] 2024-08-20T22:05:37.3216608Z ##[group]Setting up auth for fetching submodules 2024-08-20T22:05:37.3223086Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:05:37.3282233Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T22:05:37.3322068Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T22:05:37.3360744Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T22:05:37.3397244Z ##[endgroup] 2024-08-20T22:05:37.3399679Z ##[group]Fetching submodules 2024-08-20T22:05:37.3401154Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T22:05:37.3776910Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T22:05:37.4141735Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T22:05:37.4143278Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T22:05:37.4146711Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T22:05:37.4150175Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T22:05:37.4154525Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T22:05:37.4157367Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T22:05:37.4161143Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T22:05:37.4165035Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T22:05:37.4168768Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T22:05:37.4172839Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T22:05:37.4176754Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T22:05:37.4181206Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T22:05:37.4185472Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T22:05:37.4189999Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T22:05:37.4194640Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T22:05:37.4199788Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:05:37.4207716Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T22:05:37.4212435Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T22:05:37.4216972Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T22:05:37.4222267Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T22:05:37.4227030Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T22:05:37.4231994Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T22:05:37.4236906Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T22:05:37.4242025Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T22:05:37.4247089Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T22:05:37.4254038Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T22:05:37.4259081Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T22:05:37.4264723Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T22:05:37.4270068Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T22:05:37.4275693Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T22:05:37.4281200Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T22:05:37.4287093Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T22:05:37.4292567Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T22:05:37.4303332Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T22:05:37.4338067Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T22:05:37.7078865Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T22:05:37.9159564Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T22:05:38.0926041Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T22:05:38.3291599Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T22:05:40.2777451Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T22:05:51.8906538Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T22:05:52.2962986Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T22:05:52.6937268Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T22:05:53.2874499Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T22:05:54.5408580Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T22:05:56.6108777Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T22:06:02.8929902Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T22:06:04.3151840Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T22:06:06.1357845Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T22:06:07.4654967Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T22:06:07.8782274Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T22:06:08.2012800Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T22:06:09.4408088Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T22:06:09.7744546Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T22:06:10.0080577Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T22:06:11.6980504Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T22:06:12.3983332Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T22:06:13.0561694Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T22:06:18.8303220Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T22:06:20.7784739Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T22:06:24.9902352Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T22:06:25.2264894Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T22:06:35.0986831Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T22:06:35.2965250Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T22:06:35.4923808Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T22:06:36.4530594Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T22:06:36.7338711Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T22:06:37.3573079Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T22:06:37.7797982Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T22:06:37.7947927Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T22:06:37.8065461Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T22:06:37.8383684Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T22:06:37.8839282Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T22:06:39.0704834Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T22:06:39.0987237Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T22:06:39.1520763Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T22:06:39.2655395Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T22:06:39.3060090Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T22:06:39.9301552Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T22:06:40.2180624Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T22:06:40.3123671Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T22:06:40.3147214Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:06:40.3150625Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:06:40.3154510Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:06:40.3158527Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:06:40.3162742Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:06:40.3194299Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T22:06:41.4881335Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T22:06:42.1091996Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T22:06:44.2670766Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T22:06:45.4036894Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T22:06:45.7589234Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T22:06:45.8699391Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T22:06:46.3842606Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T22:06:46.4551689Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T22:06:46.4705722Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T22:06:46.6210425Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T22:06:46.6671400Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T22:06:46.7138624Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T22:06:46.7461842Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T22:06:46.7986821Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:06:46.8142803Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T22:06:46.8162944Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T22:06:46.8192511Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T22:06:59.8465541Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T22:06:59.8687862Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T22:06:59.9690123Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T22:06:59.9712743Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:06:59.9716188Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:06:59.9720281Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:06:59.9753270Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T22:07:00.5011985Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T22:07:01.7227884Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T22:07:02.8678711Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T22:07:02.8698867Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:07:02.8702559Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:07:02.8706085Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:07:02.8709732Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:07:02.8713639Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:07:02.8717586Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:07:02.8721011Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:07:02.8724869Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:07:02.8760453Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T22:07:03.7529388Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T22:07:04.1258695Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T22:07:05.3318825Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T22:07:05.6062213Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T22:07:06.2468020Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T22:07:07.3315805Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T22:07:14.2137570Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T22:07:14.6064659Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T22:07:14.6302966Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T22:07:14.6735710Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T22:07:14.6900414Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T22:07:14.6921341Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:07:14.6952317Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T22:07:15.0015784Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T22:07:15.0243306Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T22:07:15.0724327Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T22:07:15.1927029Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T22:07:15.2126331Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T22:07:15.2582382Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T22:07:15.3246523Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T22:07:15.3710710Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T22:07:15.4009424Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T22:07:15.5265577Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T22:07:16.0396272Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T22:07:16.0433499Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T22:07:16.0436680Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T22:07:16.0467995Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T22:07:16.4873713Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T22:07:17.4548000Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T22:07:17.4964415Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T22:07:17.5835770Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T22:07:17.5856565Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:07:17.5860004Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:07:17.5863435Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:07:17.5866912Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:07:17.5871847Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:07:17.5875320Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:07:17.5880641Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:07:17.5884171Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:07:17.5921571Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T22:07:18.1350164Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T22:07:19.2431544Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T22:07:19.5611718Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T22:07:25.5812169Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T22:07:25.8746120Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T22:07:26.0956697Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T22:07:26.4009290Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T22:07:33.3326467Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T22:07:33.3807626Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T22:07:33.3993752Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T22:07:33.5200082Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T22:07:33.5361717Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T22:07:33.5551364Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T22:07:33.5754480Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T22:07:33.5773462Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:07:33.5776773Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:07:33.5810049Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T22:07:35.4758472Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T22:07:36.8579797Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T22:07:36.9108212Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:07:37.5879572Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T22:07:37.6028485Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T22:07:37.9196424Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T22:07:37.9222670Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:07:37.9226238Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T22:07:37.9260011Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T22:07:38.3661709Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T22:07:39.4321876Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T22:07:39.5146223Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T22:07:39.5260677Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T22:07:39.5411341Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T22:07:39.5866152Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T22:07:39.6211744Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T22:07:39.6697079Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T22:07:39.7043846Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T22:07:39.7064993Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:07:39.7068136Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:07:39.7071671Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:07:39.7074984Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:07:39.7105106Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T22:07:40.7591534Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T22:07:41.0026895Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T22:07:42.2693484Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T22:07:43.2239332Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T22:07:43.2437509Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T22:07:43.3216632Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T22:07:43.3564176Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T22:07:43.3584666Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:07:43.3616557Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T22:07:43.5654507Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T22:07:43.5699382Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T22:07:43.6062767Z Entering 'android/libs/fbjni' 2024-08-20T22:07:43.6114177Z Entering 'third_party/FP16' 2024-08-20T22:07:43.6165693Z Entering 'third_party/FXdiv' 2024-08-20T22:07:43.6216351Z Entering 'third_party/NNPACK' 2024-08-20T22:07:43.6268835Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:07:43.6321456Z Entering 'third_party/XNNPACK' 2024-08-20T22:07:43.6389185Z Entering 'third_party/benchmark' 2024-08-20T22:07:43.6443010Z Entering 'third_party/cpp-httplib' 2024-08-20T22:07:43.6493255Z Entering 'third_party/cpuinfo' 2024-08-20T22:07:43.6544086Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:07:43.6595352Z Entering 'third_party/cutlass' 2024-08-20T22:07:43.6657027Z Entering 'third_party/eigen' 2024-08-20T22:07:43.6713580Z Entering 'third_party/fbgemm' 2024-08-20T22:07:43.6764247Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:07:43.6818392Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:07:43.6868366Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:07:43.6927396Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:07:43.6978866Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:07:43.7031725Z Entering 'third_party/flatbuffers' 2024-08-20T22:07:43.7085635Z Entering 'third_party/fmt' 2024-08-20T22:07:43.7137719Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:07:43.7189470Z Entering 'third_party/gloo' 2024-08-20T22:07:43.7239900Z Entering 'third_party/googletest' 2024-08-20T22:07:43.7290102Z Entering 'third_party/ideep' 2024-08-20T22:07:43.7338698Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:07:43.7405693Z Entering 'third_party/ittapi' 2024-08-20T22:07:43.7458171Z Entering 'third_party/kineto' 2024-08-20T22:07:43.7507468Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:07:43.7560780Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:07:43.7614728Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:07:43.7667284Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:07:43.7719305Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:07:43.7770148Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:07:43.7829404Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:07:43.7880735Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:07:43.7932887Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:07:43.7984347Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:07:43.8038320Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:07:43.8087841Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:07:43.8141969Z Entering 'third_party/mimalloc' 2024-08-20T22:07:43.8193486Z Entering 'third_party/nccl/nccl' 2024-08-20T22:07:43.8245713Z Entering 'third_party/nlohmann' 2024-08-20T22:07:43.8299170Z Entering 'third_party/onnx' 2024-08-20T22:07:43.8364193Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:07:43.8414995Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:07:43.8476616Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:07:43.8530016Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:07:43.8585744Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:07:43.8635645Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:07:43.8686024Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:07:43.8737557Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:07:43.8787422Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:07:43.8836962Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:07:43.8885361Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:07:43.8938275Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:07:43.8991534Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:07:43.9066380Z Entering 'third_party/pocketfft' 2024-08-20T22:07:43.9118393Z Entering 'third_party/protobuf' 2024-08-20T22:07:43.9171233Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:07:43.9222391Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:07:43.9279156Z Entering 'third_party/psimd' 2024-08-20T22:07:43.9330737Z Entering 'third_party/pthreadpool' 2024-08-20T22:07:43.9381355Z Entering 'third_party/pybind11' 2024-08-20T22:07:43.9433039Z Entering 'third_party/python-peachpy' 2024-08-20T22:07:43.9484493Z Entering 'third_party/sleef' 2024-08-20T22:07:43.9536281Z Entering 'third_party/tensorpipe' 2024-08-20T22:07:43.9585567Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:07:43.9635742Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:07:43.9685621Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:07:43.9738488Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:07:43.9786977Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:07:43.9862886Z ##[endgroup] 2024-08-20T22:07:43.9868511Z ##[group]Persisting credentials for submodules 2024-08-20T22:07:43.9870355Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T22:07:44.0227934Z Entering 'android/libs/fbjni' 2024-08-20T22:07:44.0295154Z Entering 'third_party/FP16' 2024-08-20T22:07:44.0362725Z Entering 'third_party/FXdiv' 2024-08-20T22:07:44.0429819Z Entering 'third_party/NNPACK' 2024-08-20T22:07:44.0500940Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:07:44.0569600Z Entering 'third_party/XNNPACK' 2024-08-20T22:07:44.0653272Z Entering 'third_party/benchmark' 2024-08-20T22:07:44.0720864Z Entering 'third_party/cpp-httplib' 2024-08-20T22:07:44.0792746Z Entering 'third_party/cpuinfo' 2024-08-20T22:07:44.0859664Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:07:44.0928623Z Entering 'third_party/cutlass' 2024-08-20T22:07:44.1003874Z Entering 'third_party/eigen' 2024-08-20T22:07:44.1072685Z Entering 'third_party/fbgemm' 2024-08-20T22:07:44.1139594Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:07:44.1206476Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:07:44.1272326Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:07:44.1346207Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:07:44.1413441Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:07:44.1483124Z Entering 'third_party/flatbuffers' 2024-08-20T22:07:44.1557040Z Entering 'third_party/fmt' 2024-08-20T22:07:44.1624224Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:07:44.1690644Z Entering 'third_party/gloo' 2024-08-20T22:07:44.1759733Z Entering 'third_party/googletest' 2024-08-20T22:07:44.1827697Z Entering 'third_party/ideep' 2024-08-20T22:07:44.1892394Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:07:44.1966870Z Entering 'third_party/ittapi' 2024-08-20T22:07:44.2037055Z Entering 'third_party/kineto' 2024-08-20T22:07:44.2101796Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:07:44.2171643Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:07:44.2240063Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:07:44.2313030Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:07:44.2379706Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:07:44.2445173Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:07:44.2517158Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:07:44.2587138Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:07:44.2654841Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:07:44.2724249Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:07:44.2794084Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:07:44.2860734Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:07:44.2930813Z Entering 'third_party/mimalloc' 2024-08-20T22:07:44.2997666Z Entering 'third_party/nccl/nccl' 2024-08-20T22:07:44.3067232Z Entering 'third_party/nlohmann' 2024-08-20T22:07:44.3136163Z Entering 'third_party/onnx' 2024-08-20T22:07:44.3216451Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:07:44.3285959Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:07:44.3358088Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:07:44.3426062Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:07:44.3492598Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:07:44.3560657Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:07:44.3628698Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:07:44.3696311Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:07:44.3762041Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:07:44.3833269Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:07:44.3898453Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:07:44.3967016Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:07:44.4036828Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:07:44.4124416Z Entering 'third_party/pocketfft' 2024-08-20T22:07:44.4194029Z Entering 'third_party/protobuf' 2024-08-20T22:07:44.4267613Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:07:44.4333608Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:07:44.4405178Z Entering 'third_party/psimd' 2024-08-20T22:07:44.4472728Z Entering 'third_party/pthreadpool' 2024-08-20T22:07:44.4540842Z Entering 'third_party/pybind11' 2024-08-20T22:07:44.4612262Z Entering 'third_party/python-peachpy' 2024-08-20T22:07:44.4678876Z Entering 'third_party/sleef' 2024-08-20T22:07:44.4748194Z Entering 'third_party/tensorpipe' 2024-08-20T22:07:44.4814401Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:07:44.4883315Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:07:44.4949905Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:07:44.5017768Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:07:44.5082883Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:07:44.5178165Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T22:07:44.5542790Z Entering 'android/libs/fbjni' 2024-08-20T22:07:44.5607922Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T22:07:44.5629310Z Entering 'third_party/FP16' 2024-08-20T22:07:44.5693214Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T22:07:44.5717875Z Entering 'third_party/FXdiv' 2024-08-20T22:07:44.5780691Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T22:07:44.5802776Z Entering 'third_party/NNPACK' 2024-08-20T22:07:44.5865034Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T22:07:44.5886912Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:07:44.5949606Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T22:07:44.5971277Z Entering 'third_party/XNNPACK' 2024-08-20T22:07:44.6033503Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T22:07:44.6070063Z Entering 'third_party/benchmark' 2024-08-20T22:07:44.6132041Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:07:44.6156877Z Entering 'third_party/cpp-httplib' 2024-08-20T22:07:44.6219190Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T22:07:44.6240948Z Entering 'third_party/cpuinfo' 2024-08-20T22:07:44.6302675Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:07:44.6324397Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:07:44.6386436Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T22:07:44.6408782Z Entering 'third_party/cutlass' 2024-08-20T22:07:44.6471148Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:07:44.6501082Z Entering 'third_party/eigen' 2024-08-20T22:07:44.6564134Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T22:07:44.6588055Z Entering 'third_party/fbgemm' 2024-08-20T22:07:44.6654178Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T22:07:44.6674235Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:07:44.6735573Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T22:07:44.6757095Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:07:44.6819099Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:07:44.6840529Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:07:44.6903810Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:07:44.6928461Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:07:44.6989839Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:07:44.7010128Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:07:44.7071949Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T22:07:44.7095071Z Entering 'third_party/flatbuffers' 2024-08-20T22:07:44.7158641Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T22:07:44.7182558Z Entering 'third_party/fmt' 2024-08-20T22:07:44.7244247Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:07:44.7265839Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:07:44.7328057Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T22:07:44.7349416Z Entering 'third_party/gloo' 2024-08-20T22:07:44.7411164Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T22:07:44.7432897Z Entering 'third_party/googletest' 2024-08-20T22:07:44.7496670Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:07:44.7517902Z Entering 'third_party/ideep' 2024-08-20T22:07:44.7580280Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T22:07:44.7599582Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:07:44.7661262Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T22:07:44.7691920Z Entering 'third_party/ittapi' 2024-08-20T22:07:44.7753510Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T22:07:44.7774357Z Entering 'third_party/kineto' 2024-08-20T22:07:44.7836543Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T22:07:44.7855976Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:07:44.7918246Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T22:07:44.7936786Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:07:44.7999060Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T22:07:44.8020859Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:07:44.8082463Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T22:07:44.8107716Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:07:44.8168301Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:07:44.8188675Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:07:44.8250952Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T22:07:44.8274400Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:07:44.8338843Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T22:07:44.8361752Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:07:44.8425379Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T22:07:44.8445802Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:07:44.8508164Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:07:44.8528818Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:07:44.8592139Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T22:07:44.8613943Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:07:44.8679145Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T22:07:44.8703625Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:07:44.8765428Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T22:07:44.8785274Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:07:44.8846743Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T22:07:44.8869965Z Entering 'third_party/mimalloc' 2024-08-20T22:07:44.8933876Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T22:07:44.8954254Z Entering 'third_party/nccl/nccl' 2024-08-20T22:07:44.9016371Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T22:07:44.9037484Z Entering 'third_party/nlohmann' 2024-08-20T22:07:44.9098426Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T22:07:44.9120888Z Entering 'third_party/onnx' 2024-08-20T22:07:44.9181923Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T22:07:44.9217346Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:07:44.9278118Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:07:44.9299839Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:07:44.9361370Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:07:44.9387636Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:07:44.9450844Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T22:07:44.9473053Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:07:44.9534105Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:07:44.9554023Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:07:44.9615753Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:07:44.9635516Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:07:44.9700260Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T22:07:44.9720919Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:07:44.9780848Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T22:07:44.9804088Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:07:44.9864052Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T22:07:44.9884369Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:07:44.9949152Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T22:07:44.9969293Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:07:45.0033549Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T22:07:45.0051708Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:07:45.0115006Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T22:07:45.0137275Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:07:45.0203619Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T22:07:45.0227271Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:07:45.0290506Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T22:07:45.0332296Z Entering 'third_party/pocketfft' 2024-08-20T22:07:45.0395131Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T22:07:45.0416797Z Entering 'third_party/protobuf' 2024-08-20T22:07:45.0478557Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T22:07:45.0502363Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:07:45.0563190Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:07:45.0582836Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:07:45.0649815Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:07:45.0674359Z Entering 'third_party/psimd' 2024-08-20T22:07:45.0738803Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T22:07:45.0760416Z Entering 'third_party/pthreadpool' 2024-08-20T22:07:45.0821293Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T22:07:45.0842780Z Entering 'third_party/pybind11' 2024-08-20T22:07:45.0903636Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:07:45.0925468Z Entering 'third_party/python-peachpy' 2024-08-20T22:07:45.0985753Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T22:07:45.1008014Z Entering 'third_party/sleef' 2024-08-20T22:07:45.1070031Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T22:07:45.1091836Z Entering 'third_party/tensorpipe' 2024-08-20T22:07:45.1158097Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T22:07:45.1178379Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:07:45.1239912Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:07:45.1259344Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:07:45.1321024Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T22:07:45.1340350Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:07:45.1406405Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T22:07:45.1427304Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:07:45.1487461Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:07:45.1507375Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:07:45.1567734Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T22:07:45.2312974Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T22:07:45.2679616Z Entering 'android/libs/fbjni' 2024-08-20T22:07:45.2732129Z Entering 'third_party/FP16' 2024-08-20T22:07:45.2790282Z Entering 'third_party/FXdiv' 2024-08-20T22:07:45.2844303Z Entering 'third_party/NNPACK' 2024-08-20T22:07:45.2896754Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:07:45.2951427Z Entering 'third_party/XNNPACK' 2024-08-20T22:07:45.3019798Z Entering 'third_party/benchmark' 2024-08-20T22:07:45.3076147Z Entering 'third_party/cpp-httplib' 2024-08-20T22:07:45.3128798Z Entering 'third_party/cpuinfo' 2024-08-20T22:07:45.3181457Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:07:45.3234498Z Entering 'third_party/cutlass' 2024-08-20T22:07:45.3294368Z Entering 'third_party/eigen' 2024-08-20T22:07:45.3348372Z Entering 'third_party/fbgemm' 2024-08-20T22:07:45.3399954Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:07:45.3450047Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:07:45.3500523Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:07:45.3557039Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:07:45.3607017Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:07:45.3659705Z Entering 'third_party/flatbuffers' 2024-08-20T22:07:45.3714789Z Entering 'third_party/fmt' 2024-08-20T22:07:45.3766363Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:07:45.3818896Z Entering 'third_party/gloo' 2024-08-20T22:07:45.3873066Z Entering 'third_party/googletest' 2024-08-20T22:07:45.3927890Z Entering 'third_party/ideep' 2024-08-20T22:07:45.3977306Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:07:45.4036773Z Entering 'third_party/ittapi' 2024-08-20T22:07:45.4089015Z Entering 'third_party/kineto' 2024-08-20T22:07:45.4141043Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:07:45.4189898Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:07:45.4242516Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:07:45.4293995Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:07:45.4346925Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:07:45.4399554Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:07:45.4453750Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:07:45.4506356Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:07:45.4558620Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:07:45.4610891Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:07:45.4665207Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:07:45.4715566Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:07:45.4768068Z Entering 'third_party/mimalloc' 2024-08-20T22:07:45.4820998Z Entering 'third_party/nccl/nccl' 2024-08-20T22:07:45.4874174Z Entering 'third_party/nlohmann' 2024-08-20T22:07:45.4929698Z Entering 'third_party/onnx' 2024-08-20T22:07:45.4999983Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:07:45.5051837Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:07:45.5110195Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:07:45.5162722Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:07:45.5213293Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:07:45.5263788Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:07:45.5313956Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:07:45.5366772Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:07:45.5415474Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:07:45.5465671Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:07:45.5516864Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:07:45.5569467Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:07:45.5630550Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:07:45.5703377Z Entering 'third_party/pocketfft' 2024-08-20T22:07:45.5755108Z Entering 'third_party/protobuf' 2024-08-20T22:07:45.5829728Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:07:45.5880128Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:07:45.5938640Z Entering 'third_party/psimd' 2024-08-20T22:07:45.5989831Z Entering 'third_party/pthreadpool' 2024-08-20T22:07:45.6044457Z Entering 'third_party/pybind11' 2024-08-20T22:07:45.6099628Z Entering 'third_party/python-peachpy' 2024-08-20T22:07:45.6151809Z Entering 'third_party/sleef' 2024-08-20T22:07:45.6204452Z Entering 'third_party/tensorpipe' 2024-08-20T22:07:45.6256595Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:07:45.6306303Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:07:45.6355604Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:07:45.6405915Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:07:45.6454797Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:07:45.6534666Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T22:07:45.6902869Z Entering 'android/libs/fbjni' 2024-08-20T22:07:45.6958573Z Entering 'third_party/FP16' 2024-08-20T22:07:45.7011076Z Entering 'third_party/FXdiv' 2024-08-20T22:07:45.7062836Z Entering 'third_party/NNPACK' 2024-08-20T22:07:45.7116871Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:07:45.7168890Z Entering 'third_party/XNNPACK' 2024-08-20T22:07:45.7237048Z Entering 'third_party/benchmark' 2024-08-20T22:07:45.7289990Z Entering 'third_party/cpp-httplib' 2024-08-20T22:07:45.7344103Z Entering 'third_party/cpuinfo' 2024-08-20T22:07:45.7399544Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:07:45.7450640Z Entering 'third_party/cutlass' 2024-08-20T22:07:45.7514043Z Entering 'third_party/eigen' 2024-08-20T22:07:45.7567562Z Entering 'third_party/fbgemm' 2024-08-20T22:07:45.7619523Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:07:45.7670161Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:07:45.7721042Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:07:45.7777158Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:07:45.7826126Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:07:45.7877548Z Entering 'third_party/flatbuffers' 2024-08-20T22:07:45.7931808Z Entering 'third_party/fmt' 2024-08-20T22:07:45.7995280Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:07:45.8046766Z Entering 'third_party/gloo' 2024-08-20T22:07:45.8099193Z Entering 'third_party/googletest' 2024-08-20T22:07:45.8151569Z Entering 'third_party/ideep' 2024-08-20T22:07:45.8200612Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:07:45.8259901Z Entering 'third_party/ittapi' 2024-08-20T22:07:45.8310183Z Entering 'third_party/kineto' 2024-08-20T22:07:45.8360164Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:07:45.8410404Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:07:45.8462214Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:07:45.8518300Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:07:45.8569769Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:07:45.8620372Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:07:45.8675608Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:07:45.8726933Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:07:45.8778536Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:07:45.8831074Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:07:45.8886904Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:07:45.8939001Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:07:45.8994839Z Entering 'third_party/mimalloc' 2024-08-20T22:07:45.9046821Z Entering 'third_party/nccl/nccl' 2024-08-20T22:07:45.9098404Z Entering 'third_party/nlohmann' 2024-08-20T22:07:45.9150795Z Entering 'third_party/onnx' 2024-08-20T22:07:45.9219115Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:07:45.9270521Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:07:45.9328475Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:07:45.9386470Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:07:45.9435911Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:07:45.9486075Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:07:45.9534396Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:07:45.9585614Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:07:45.9634020Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:07:45.9682523Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:07:45.9729857Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:07:45.9784144Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:07:45.9838282Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:07:45.9909657Z Entering 'third_party/pocketfft' 2024-08-20T22:07:45.9962843Z Entering 'third_party/protobuf' 2024-08-20T22:07:46.0020803Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:07:46.0071460Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:07:46.0125189Z Entering 'third_party/psimd' 2024-08-20T22:07:46.0176847Z Entering 'third_party/pthreadpool' 2024-08-20T22:07:46.0229071Z Entering 'third_party/pybind11' 2024-08-20T22:07:46.0279781Z Entering 'third_party/python-peachpy' 2024-08-20T22:07:46.0336264Z Entering 'third_party/sleef' 2024-08-20T22:07:46.0387720Z Entering 'third_party/tensorpipe' 2024-08-20T22:07:46.0439849Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:07:46.0490060Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:07:46.0541211Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:07:46.0590081Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:07:46.0637605Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:07:46.0710750Z ##[endgroup] 2024-08-20T22:07:46.0762635Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T22:07:46.0800158Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T22:07:46.1028288Z Prepare all required actions 2024-08-20T22:07:46.1028782Z Getting action download info 2024-08-20T22:07:46.2862045Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T22:07:46.2862486Z env: 2024-08-20T22:07:46.2862814Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:46.2863163Z ##[endgroup] 2024-08-20T22:07:46.2924376Z ##[group]Run set -euo pipefail 2024-08-20T22:07:46.2924806Z set -euo pipefail 2024-08-20T22:07:46.2925205Z function get_ec2_metadata() { 2024-08-20T22:07:46.2925749Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T22:07:46.2926633Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T22:07:46.2927399Z  category=$1 2024-08-20T22:07:46.2927931Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T22:07:46.2928579Z  runner_name_str=i-02d1d9ee19baacafb 2024-08-20T22:07:46.2929073Z  if [[ -f /.inarc ]]; then 2024-08-20T22:07:46.2929573Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T22:07:46.2930148Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T22:07:46.2930839Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T22:07:46.2931450Z  else 2024-08-20T22:07:46.2931941Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T22:07:46.2932548Z  fi 2024-08-20T22:07:46.2932864Z } 2024-08-20T22:07:46.2933222Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T22:07:46.2933811Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T22:07:46.2934468Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T22:07:46.2935037Z echo "system info $(uname -a)" 2024-08-20T22:07:46.2945262Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:46.2945752Z env: 2024-08-20T22:07:46.2946043Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:46.2946393Z ##[endgroup] 2024-08-20T22:07:46.3066300Z ami-id: ami-06c68f701d8090592 2024-08-20T22:07:46.3125020Z instance-id: i-02d1d9ee19baacafb 2024-08-20T22:07:46.3182280Z instance-type: g5.4xlarge 2024-08-20T22:07:46.3197641Z system info Linux ip-10-0-46-136.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T22:07:46.3221009Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:07:46.3221920Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:07:46.3231179Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:46.3231687Z env: 2024-08-20T22:07:46.3231973Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:46.3232327Z ##[endgroup] 2024-08-20T22:07:46.3306481Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T22:07:46.3307093Z if systemctl is-active --quiet docker; then 2024-08-20T22:07:46.3307641Z  echo "Docker daemon is running..."; 2024-08-20T22:07:46.3308100Z else 2024-08-20T22:07:46.3308592Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T22:07:46.3309343Z fi 2024-08-20T22:07:46.3317460Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:46.3317948Z env: 2024-08-20T22:07:46.3318231Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:46.3318576Z ##[endgroup] 2024-08-20T22:07:46.3413305Z Docker daemon is running... 2024-08-20T22:07:46.3470468Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:07:46.3471029Z with: 2024-08-20T22:07:46.3471301Z shell: bash 2024-08-20T22:07:46.3471597Z timeout_minutes: 5 2024-08-20T22:07:46.3471922Z max_attempts: 3 2024-08-20T22:07:46.3472240Z retry_wait_seconds: 30 2024-08-20T22:07:46.3475370Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T22:07:46.3478354Z polling_interval_seconds: 1 2024-08-20T22:07:46.3478733Z warning_on_retry: true 2024-08-20T22:07:46.3479094Z continue_on_error: false 2024-08-20T22:07:46.3479441Z env: 2024-08-20T22:07:46.3479724Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:46.3480086Z AWS_RETRY_MODE: standard 2024-08-20T22:07:46.3480444Z AWS_MAX_ATTEMPTS: 5 2024-08-20T22:07:46.3480796Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T22:07:46.3481181Z ##[endgroup] 2024-08-20T22:07:47.5094707Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:07:47.5095766Z Configure a credential helper to remove this warning. See 2024-08-20T22:07:47.5096856Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:07:47.5097586Z 2024-08-20T22:07:47.5097744Z Login Succeeded 2024-08-20T22:07:48.4051536Z Command completed after 1 attempt(s). 2024-08-20T22:07:48.4121420Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:07:48.4122203Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:07:48.4122916Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:07:48.4133779Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:48.4134278Z env: 2024-08-20T22:07:48.4134565Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:48.4134924Z ##[endgroup] 2024-08-20T22:07:48.4230991Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:07:48.4231759Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:07:48.4232347Z # shellcheck disable=SC2046 2024-08-20T22:07:48.4232804Z docker stop $(docker ps -q) || true 2024-08-20T22:07:48.4233292Z # Prune all of the docker images 2024-08-20T22:07:48.4233737Z docker system prune -af 2024-08-20T22:07:48.4242680Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:48.4243207Z env: 2024-08-20T22:07:48.4243484Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:48.4243835Z ##[endgroup] 2024-08-20T22:07:48.4562106Z "docker stop" requires at least 1 argument. 2024-08-20T22:07:48.4562932Z See 'docker stop --help'. 2024-08-20T22:07:48.4563166Z 2024-08-20T22:07:48.4563387Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T22:07:48.4563759Z 2024-08-20T22:07:48.4563906Z Stop one or more running containers 2024-08-20T22:07:48.4750318Z Total reclaimed space: 0B 2024-08-20T22:07:48.4822787Z ##[group]Run set +e 2024-08-20T22:07:48.4823156Z set +e 2024-08-20T22:07:48.4823464Z set -x 2024-08-20T22:07:48.4823767Z  2024-08-20T22:07:48.4824099Z PT_DOMAIN=download.pytorch.org 2024-08-20T22:07:48.4825070Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T22:07:48.4826168Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T22:07:48.4826931Z # one is returned at random 2024-08-20T22:07:48.4827478Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T22:07:48.4827985Z  2024-08-20T22:07:48.4828306Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:07:48.4828929Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T22:07:48.4829866Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T22:07:48.4830422Z  2024-08-20T22:07:48.4830749Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:07:48.4831301Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T22:07:48.4831816Z  exit 1 2024-08-20T22:07:48.4832134Z  fi 2024-08-20T22:07:48.4832419Z fi 2024-08-20T22:07:48.4832698Z  2024-08-20T22:07:48.4833055Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T22:07:48.4833566Z  # Clean up any old records first 2024-08-20T22:07:48.4834071Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T22:07:48.4834522Z fi 2024-08-20T22:07:48.4834792Z  2024-08-20T22:07:48.4835229Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T22:07:48.4835777Z cat /etc/hosts 2024-08-20T22:07:48.4845124Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:48.4845620Z env: 2024-08-20T22:07:48.4845910Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:48.4846258Z ##[endgroup] 2024-08-20T22:07:48.4875103Z + PT_DOMAIN=download.pytorch.org 2024-08-20T22:07:48.4881478Z ++ dig -4 +short download.pytorch.org 2024-08-20T22:07:48.4881948Z ++ tail -n1 2024-08-20T22:07:48.5174001Z + RESOLVED_IP=18.160.10.28 2024-08-20T22:07:48.5174561Z + '[' -z 18.160.10.28 ']' 2024-08-20T22:07:48.5175011Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T22:07:48.5192748Z + echo '18.160.10.28 download.pytorch.org' 2024-08-20T22:07:48.5193944Z + sudo tee -a /etc/hosts 2024-08-20T22:07:48.7678215Z 18.160.10.28 download.pytorch.org 2024-08-20T22:07:48.7701243Z + cat /etc/hosts 2024-08-20T22:07:48.7711796Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T22:07:48.7724274Z ::1 localhost6 localhost6.localdomain6 2024-08-20T22:07:48.7724993Z 18.160.10.28 download.pytorch.org 2024-08-20T22:07:48.7883638Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T22:07:48.7884235Z with: 2024-08-20T22:07:48.7885328Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:48.7886549Z docker-build-dir: .ci/docker 2024-08-20T22:07:48.7886941Z working-directory: . 2024-08-20T22:07:48.7887409Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:48.7887931Z force-push: false 2024-08-20T22:07:48.7888234Z env: 2024-08-20T22:07:48.7888509Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:48.7888846Z ##[endgroup] 2024-08-20T22:07:48.7908428Z ##[group]Run set -ex 2024-08-20T22:07:48.7908801Z set -ex 2024-08-20T22:07:48.7909097Z  2024-08-20T22:07:48.7909656Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T22:07:48.7910663Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T22:07:48.7911467Z # job could then download the pre-built image as usual 2024-08-20T22:07:48.7912209Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T22:07:48.7912885Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:07:48.7913694Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:07:48.7914255Z  2024-08-20T22:07:48.7914765Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T22:07:48.7915393Z  exit 0 2024-08-20T22:07:48.7915700Z else 2024-08-20T22:07:48.7916066Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T22:07:48.7916525Z fi 2024-08-20T22:07:48.7916809Z  2024-08-20T22:07:48.7917283Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T22:07:48.7918124Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T22:07:48.7918902Z  # use it as it is, but first let's extract the tag 2024-08-20T22:07:48.7919600Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T22:07:48.7920324Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:07:48.7921033Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:07:48.7921587Z else 2024-08-20T22:07:48.7922026Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T22:07:48.7922688Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:07:48.7923579Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:07:48.7924323Z fi 2024-08-20T22:07:48.7935595Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:48.7936091Z env: 2024-08-20T22:07:48.7936372Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:48.7936721Z REPO_NAME: pytorch 2024-08-20T22:07:48.7937852Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:48.7939064Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:07:48.7939707Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:48.7940218Z ##[endgroup] 2024-08-20T22:07:48.7970220Z + [[ ! -d .ci/docker ]] 2024-08-20T22:07:48.7970881Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T22:07:48.7971635Z + echo skip=false 2024-08-20T22:07:48.7973761Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T22:07:48.7977578Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:48.7979203Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T22:07:48.8006685Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:48.8007775Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:48.8009714Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:48.8042955Z ##[group]Run set +e 2024-08-20T22:07:48.8043433Z set +e 2024-08-20T22:07:48.8043865Z set -x 2024-08-20T22:07:48.8044250Z  2024-08-20T22:07:48.8044640Z login() { 2024-08-20T22:07:48.8045438Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:07:48.8046202Z } 2024-08-20T22:07:48.8046587Z  2024-08-20T22:07:48.8046995Z retry () { 2024-08-20T22:07:48.8047470Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:07:48.8047992Z } 2024-08-20T22:07:48.8048399Z  2024-08-20T22:07:48.8048805Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:07:48.8049476Z  2024-08-20T22:07:48.8050059Z # Check if image already exists, if it does then skip building it 2024-08-20T22:07:48.8050875Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T22:07:48.8051465Z  exit 0 2024-08-20T22:07:48.8051897Z fi 2024-08-20T22:07:48.8052249Z  2024-08-20T22:07:48.8052886Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T22:07:48.8053872Z # be empty. The default action would be to continue rebuild the image 2024-08-20T22:07:48.8054707Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T22:07:48.8055432Z  # if we're on the base branch then use the parent commit 2024-08-20T22:07:48.8056193Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T22:07:48.8056795Z else 2024-08-20T22:07:48.8057406Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T22:07:48.8058170Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T22:07:48.8058771Z fi 2024-08-20T22:07:48.8059184Z  2024-08-20T22:07:48.8059744Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T22:07:48.8060306Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:07:48.8060945Z  2024-08-20T22:07:48.8061689Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T22:07:48.8062542Z  exit 0 2024-08-20T22:07:48.8063027Z fi 2024-08-20T22:07:48.8063404Z  2024-08-20T22:07:48.8064018Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T22:07:48.8065182Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T22:07:48.8066046Z  exit 1 2024-08-20T22:07:48.8066442Z fi 2024-08-20T22:07:48.8066902Z  2024-08-20T22:07:48.8067450Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T22:07:48.8068516Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T22:07:48.8069521Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T22:07:48.8070580Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T22:07:48.8071789Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T22:07:48.8072704Z fi 2024-08-20T22:07:48.8073177Z  2024-08-20T22:07:48.8073617Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:07:48.8082067Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:48.8082625Z env: 2024-08-20T22:07:48.8083066Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:48.8083544Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:07:48.8084157Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:07:48.8085487Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:48.8086871Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:48.8087624Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:48.8088330Z ##[endgroup] 2024-08-20T22:07:48.8118431Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:48.8119242Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:48.8120648Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:07:48.8123283Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:49.3455536Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:07:49.3456959Z Configure a credential helper to remove this warning. See 2024-08-20T22:07:49.3458271Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:07:49.3458817Z 2024-08-20T22:07:49.3459086Z Login Succeeded 2024-08-20T22:07:49.3481621Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:49.5829806Z { 2024-08-20T22:07:49.5831083Z "schemaVersion": 2, 2024-08-20T22:07:49.5833039Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T22:07:49.5833719Z "config": { 2024-08-20T22:07:49.5835089Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T22:07:49.5836009Z "size": 48711, 2024-08-20T22:07:49.5836702Z "digest": "sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb" 2024-08-20T22:07:49.5837755Z }, 2024-08-20T22:07:49.5838161Z "layers": [ 2024-08-20T22:07:49.5838973Z { 2024-08-20T22:07:49.5839852Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5840503Z "size": 28580681, 2024-08-20T22:07:49.5841601Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-20T22:07:49.5842318Z }, 2024-08-20T22:07:49.5843100Z { 2024-08-20T22:07:49.5843744Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5844414Z "size": 7943451, 2024-08-20T22:07:49.5845133Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-08-20T22:07:49.5845848Z }, 2024-08-20T22:07:49.5846507Z { 2024-08-20T22:07:49.5847177Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5847946Z "size": 55728572, 2024-08-20T22:07:49.5849153Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-08-20T22:07:49.5849964Z }, 2024-08-20T22:07:49.5850355Z { 2024-08-20T22:07:49.5851190Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5851979Z "size": 186, 2024-08-20T22:07:49.5853718Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-08-20T22:07:49.5854468Z }, 2024-08-20T22:07:49.5855043Z { 2024-08-20T22:07:49.5855983Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5856747Z "size": 6886, 2024-08-20T22:07:49.5857664Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-08-20T22:07:49.5858522Z }, 2024-08-20T22:07:49.5859647Z { 2024-08-20T22:07:49.5860334Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5861303Z "size": 1329001756, 2024-08-20T22:07:49.5862183Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-08-20T22:07:49.5863695Z }, 2024-08-20T22:07:49.5864205Z { 2024-08-20T22:07:49.5864921Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5865826Z "size": 62501, 2024-08-20T22:07:49.5866474Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-08-20T22:07:49.5867152Z }, 2024-08-20T22:07:49.5867583Z { 2024-08-20T22:07:49.5868124Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5868823Z "size": 1684, 2024-08-20T22:07:49.5869433Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-08-20T22:07:49.5870139Z }, 2024-08-20T22:07:49.5870539Z { 2024-08-20T22:07:49.5871175Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5871788Z "size": 1523, 2024-08-20T22:07:49.5872516Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-08-20T22:07:49.5873284Z }, 2024-08-20T22:07:49.5873619Z { 2024-08-20T22:07:49.5874228Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5875067Z "size": 2528295403, 2024-08-20T22:07:49.5875749Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-08-20T22:07:49.5876488Z }, 2024-08-20T22:07:49.5876830Z { 2024-08-20T22:07:49.5877372Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5878141Z "size": 86016, 2024-08-20T22:07:49.5878742Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-08-20T22:07:49.5879480Z }, 2024-08-20T22:07:49.5879875Z { 2024-08-20T22:07:49.5880363Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5881041Z "size": 1822, 2024-08-20T22:07:49.5881733Z "digest": "sha256:ac23c5fcc1ef310e9cd0d138b180281188e20a3e28c51cd14751622e3c7d92ea" 2024-08-20T22:07:49.5882467Z }, 2024-08-20T22:07:49.5882768Z { 2024-08-20T22:07:49.5883401Z + exit 0 2024-08-20T22:07:49.5883964Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5884746Z "size": 246761563, 2024-08-20T22:07:49.5885373Z "digest": "sha256:ca426ffc9d2bd051ad1ff5c4f02fd5d9be161e70953cc37882c815080c708f4d" 2024-08-20T22:07:49.5886123Z }, 2024-08-20T22:07:49.5886517Z { 2024-08-20T22:07:49.5887063Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5887667Z "size": 544, 2024-08-20T22:07:49.5888328Z "digest": "sha256:4373aa743caf6660c671295b837048c8350a2165f2336dc3b7b6c96462579e50" 2024-08-20T22:07:49.5889051Z }, 2024-08-20T22:07:49.5889349Z { 2024-08-20T22:07:49.5889944Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5890604Z "size": 1258, 2024-08-20T22:07:49.5891222Z "digest": "sha256:75b2abb16c685854c017d0cf75c626526c956734c656d9e78fdd4520317e9b3c" 2024-08-20T22:07:49.5892018Z }, 2024-08-20T22:07:49.5892383Z { 2024-08-20T22:07:49.5892914Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5894005Z "size": 484, 2024-08-20T22:07:49.5894595Z "digest": "sha256:ef1e3ef1e85f0124270b8506721219150e230184181f8207d64463bf5c67e656" 2024-08-20T22:07:49.5895287Z }, 2024-08-20T22:07:49.5895702Z { 2024-08-20T22:07:49.5896228Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5896834Z "size": 91711766, 2024-08-20T22:07:49.5897551Z "digest": "sha256:0bf61114d37b2d09065aa6852b1e78a32580a1ff011dd5d23f03b0f5c4e710af" 2024-08-20T22:07:49.5898255Z }, 2024-08-20T22:07:49.5898551Z { 2024-08-20T22:07:49.5899215Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5900162Z "size": 3390, 2024-08-20T22:07:49.5900881Z "digest": "sha256:2df849f72ba4258df10c9c45ace21a66d93b73196e78e96f12c13098822f60e2" 2024-08-20T22:07:49.5901554Z }, 2024-08-20T22:07:49.5901898Z { 2024-08-20T22:07:49.5902496Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5903179Z "size": 1909, 2024-08-20T22:07:49.5903763Z "digest": "sha256:34c47d63c5de27418739aa048b4dd781c3006d65691e9b903c95304d0509d89c" 2024-08-20T22:07:49.5904540Z }, 2024-08-20T22:07:49.5904886Z { 2024-08-20T22:07:49.5905398Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5906174Z "size": 701, 2024-08-20T22:07:49.5906806Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:07:49.5907540Z }, 2024-08-20T22:07:49.5907910Z { 2024-08-20T22:07:49.5908430Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5909096Z "size": 2864254879, 2024-08-20T22:07:49.5909819Z "digest": "sha256:6efb96fc87ea1bca54876d85331e5585e28f325a1d60ee1aef7811eabb74f933" 2024-08-20T22:07:49.5910485Z }, 2024-08-20T22:07:49.5910846Z { 2024-08-20T22:07:49.5911411Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5912073Z "size": 380, 2024-08-20T22:07:49.5912735Z "digest": "sha256:b119c4fc6c58d9e72744201ff4850b34184a94eb3566031b7d36e87ecce950a6" 2024-08-20T22:07:49.5913609Z }, 2024-08-20T22:07:49.5913967Z { 2024-08-20T22:07:49.5914444Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5915157Z "size": 12900, 2024-08-20T22:07:49.5915795Z "digest": "sha256:33b639226eeefa41d7bb26a8ecc20d271ceef45571dabfd80aaee3634926381b" 2024-08-20T22:07:49.5916519Z }, 2024-08-20T22:07:49.5916929Z { 2024-08-20T22:07:49.5917448Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5918051Z "size": 803, 2024-08-20T22:07:49.5918741Z "digest": "sha256:d3547bd43146930aa830d54562c98aa678e355cf5f84527e9717eb72fffe8ca3" 2024-08-20T22:07:49.5919504Z }, 2024-08-20T22:07:49.5919908Z { 2024-08-20T22:07:49.5920387Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5921016Z "size": 106, 2024-08-20T22:07:49.5921707Z "digest": "sha256:5645520256fa6d7f2270608eaed53849a131d03fbf4c5d4472c9cfde1a439e47" 2024-08-20T22:07:49.5922411Z }, 2024-08-20T22:07:49.5922717Z { 2024-08-20T22:07:49.5923338Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5924001Z "size": 503, 2024-08-20T22:07:49.5924577Z "digest": "sha256:829ed7d91e3818d0e11834c71551117c7aeec0a51d798e93d641faa0c02582cb" 2024-08-20T22:07:49.5925350Z }, 2024-08-20T22:07:49.5925689Z { 2024-08-20T22:07:49.5926306Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5926960Z "size": 121477267, 2024-08-20T22:07:49.5927605Z "digest": "sha256:385d0421d67c533d628985c453a186028a61f6ad4209ae519ec3d4ecea661d3e" 2024-08-20T22:07:49.5928315Z }, 2024-08-20T22:07:49.5928724Z { 2024-08-20T22:07:49.5929203Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5929853Z "size": 109, 2024-08-20T22:07:49.5930532Z "digest": "sha256:7cbfce999d76851e07dd71056a6d852629d789ccfd46be44cc405b451fb8bb58" 2024-08-20T22:07:49.5931219Z }, 2024-08-20T22:07:49.5931692Z { 2024-08-20T22:07:49.5932283Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5933058Z "size": 490, 2024-08-20T22:07:49.5933643Z "digest": "sha256:6658e558ebe496f0d628947534b0ea1e44808f84f2dfd8a9a704e8aa5329d791" 2024-08-20T22:07:49.5934391Z }, 2024-08-20T22:07:49.5934752Z { 2024-08-20T22:07:49.5935265Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5935904Z "size": 297, 2024-08-20T22:07:49.5936542Z "digest": "sha256:33444a5a7e01a3010fd1cc16b32a66892a08600d7f700747d23faf3c36624f3a" 2024-08-20T22:07:49.5937340Z }, 2024-08-20T22:07:49.5937718Z { 2024-08-20T22:07:49.5938242Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5938878Z "size": 103, 2024-08-20T22:07:49.5939782Z "digest": "sha256:7c47122f4e09b47e891b4ae9e499a35a4eb46faacae87d0e946615adda6e0706" 2024-08-20T22:07:49.5940453Z }, 2024-08-20T22:07:49.5940789Z { 2024-08-20T22:07:49.5941398Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5942035Z "size": 1473, 2024-08-20T22:07:49.5942719Z "digest": "sha256:1811ff4d3556ac255604f346d879528aff2aadd7652c2d39baf4fe6eaa60aec3" 2024-08-20T22:07:49.5943472Z }, 2024-08-20T22:07:49.5943809Z { 2024-08-20T22:07:49.5944402Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5945017Z "size": 594729516, 2024-08-20T22:07:49.5945667Z "digest": "sha256:7754ee068b33f760d9c320c26a1a3935247ace0c8c463a976105c1586a43fb1c" 2024-08-20T22:07:49.5946533Z }, 2024-08-20T22:07:49.5946848Z { 2024-08-20T22:07:49.5947393Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5948083Z "size": 164, 2024-08-20T22:07:49.5948697Z "digest": "sha256:0479fbac3361713c075bf788e778117917dcacdd1ff71ed79a789dd2016a1714" 2024-08-20T22:07:49.5949371Z }, 2024-08-20T22:07:49.5949753Z { 2024-08-20T22:07:49.5950274Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5951041Z "size": 564, 2024-08-20T22:07:49.5951669Z "digest": "sha256:f1d6874dee9c34d154278d2b6935d04447f07fa89d25d4a09398234040d8a7e8" 2024-08-20T22:07:49.5952391Z }, 2024-08-20T22:07:49.5952726Z { 2024-08-20T22:07:49.5953319Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5953984Z "size": 35874203, 2024-08-20T22:07:49.5954630Z "digest": "sha256:afc429e87b593a64f3e0a1be51e09e966ea1aa71e6e8a0bec4edc9f2f7bc1c05" 2024-08-20T22:07:49.5955385Z }, 2024-08-20T22:07:49.5955701Z { 2024-08-20T22:07:49.5956223Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5956936Z "size": 104, 2024-08-20T22:07:49.5957541Z "digest": "sha256:019620471c2592cd22437fe8ed1a94c784114cf7445c5c2ab36dd0bae727fc66" 2024-08-20T22:07:49.5958188Z }, 2024-08-20T22:07:49.5958588Z { 2024-08-20T22:07:49.5959096Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5959811Z "size": 425, 2024-08-20T22:07:49.5960495Z "digest": "sha256:27aeb1b2cd5e28d8f0e54fc1090baf88abc29ebf5fce088b493fcae99c9550d1" 2024-08-20T22:07:49.5961213Z }, 2024-08-20T22:07:49.5961550Z { 2024-08-20T22:07:49.5962108Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5962751Z "size": 20262183, 2024-08-20T22:07:49.5963466Z "digest": "sha256:829c0e19f4b7797e9120a86172fabfd710f9b3971ffd714a616931a3fadc7192" 2024-08-20T22:07:49.5964208Z }, 2024-08-20T22:07:49.5964509Z { 2024-08-20T22:07:49.5965047Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5965729Z "size": 438, 2024-08-20T22:07:49.5966349Z "digest": "sha256:560cd6122a3868e7e82e75eb103d182edf6512114d61ccdfae099af00cb16b4c" 2024-08-20T22:07:49.5967107Z }, 2024-08-20T22:07:49.5967514Z { 2024-08-20T22:07:49.5968033Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5968655Z "size": 701, 2024-08-20T22:07:49.5969332Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:07:49.5970067Z }, 2024-08-20T22:07:49.5970444Z { 2024-08-20T22:07:49.5970924Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5971588Z "size": 141, 2024-08-20T22:07:49.5972271Z "digest": "sha256:5a4eb19d72198a01f3babd4711c40fcf7c6e5cb9d5cfedc507b0061408d47097" 2024-08-20T22:07:49.5973043Z }, 2024-08-20T22:07:49.5973444Z { 2024-08-20T22:07:49.5974019Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5974795Z "size": 135, 2024-08-20T22:07:49.5975415Z "digest": "sha256:da6a838a66c72b5de9a5590c233779880ea901c1264940a1c618825d2501ffd1" 2024-08-20T22:07:49.5976137Z }, 2024-08-20T22:07:49.5976497Z { 2024-08-20T22:07:49.5977016Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5977673Z "size": 4838068375, 2024-08-20T22:07:49.5978342Z "digest": "sha256:cbb79a7053537bed2d80b77198ea552b985563b11d3f8616593c6ab575a3a090" 2024-08-20T22:07:49.5979041Z }, 2024-08-20T22:07:49.5979570Z { 2024-08-20T22:07:49.5980060Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5980776Z "size": 194, 2024-08-20T22:07:49.5981481Z "digest": "sha256:4d4fa6802ab3e6b36d8d4adb650046e9370c4a16a19876970559ef8a312ca855" 2024-08-20T22:07:49.5982176Z }, 2024-08-20T22:07:49.5982475Z { 2024-08-20T22:07:49.5983083Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5983710Z "size": 564, 2024-08-20T22:07:49.5984300Z "digest": "sha256:0f085d4fa279366fcd52d303b7ad8abddc5962a8872174b8109f3f5d5ca5f2cf" 2024-08-20T22:07:49.5985075Z }, 2024-08-20T22:07:49.5985412Z { 2024-08-20T22:07:49.5985947Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5986597Z "size": 43162921, 2024-08-20T22:07:49.5987328Z "digest": "sha256:4a0412373305517489d891a4749e6d2b80b4aeb7df3b8adca0ed0abb2f3e5300" 2024-08-20T22:07:49.5988146Z }, 2024-08-20T22:07:49.5988638Z { 2024-08-20T22:07:49.5989134Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5989794Z "size": 106, 2024-08-20T22:07:49.5990459Z "digest": "sha256:302c8c1779444e54919f83e91c6d5e1fdea653fc899be802b3a2a34d34d99d5d" 2024-08-20T22:07:49.5991169Z }, 2024-08-20T22:07:49.5991473Z { 2024-08-20T22:07:49.5992038Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5992702Z "size": 1380, 2024-08-20T22:07:49.5993530Z "digest": "sha256:02970347dff622ee45629bc28788716d00528464f71f88ec7b74ae5f44a486e2" 2024-08-20T22:07:49.5994378Z }, 2024-08-20T22:07:49.5994744Z { 2024-08-20T22:07:49.5995307Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5995902Z "size": 701, 2024-08-20T22:07:49.5996548Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:07:49.5997307Z }, 2024-08-20T22:07:49.5997661Z { 2024-08-20T22:07:49.5998144Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.5998827Z "size": 139, 2024-08-20T22:07:49.5999474Z "digest": "sha256:916c5e8580164926a3aec87e88a5d1c2d02756e49fdeed4198e1e6304efa2fd3" 2024-08-20T22:07:49.6000171Z }, 2024-08-20T22:07:49.6000533Z { 2024-08-20T22:07:49.6001147Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6001779Z "size": 118, 2024-08-20T22:07:49.6002449Z "digest": "sha256:c42b3602a1fb4df5111224650f15adffb025979ae9c614807379ea84e189533b" 2024-08-20T22:07:49.6003219Z }, 2024-08-20T22:07:49.6003559Z { 2024-08-20T22:07:49.6004146Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6004746Z "size": 4324672728, 2024-08-20T22:07:49.6005405Z "digest": "sha256:11fad972ef6cb9d2dc236504b0ee5b6e5ea996e4576ece0ed3262fbb9ee2de60" 2024-08-20T22:07:49.6006191Z }, 2024-08-20T22:07:49.6006531Z { 2024-08-20T22:07:49.6007009Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6007790Z "size": 174, 2024-08-20T22:07:49.6008412Z "digest": "sha256:f957c16244819132b478272aaf24231699c4fcbd495c229a574f94daab8d6b86" 2024-08-20T22:07:49.6009065Z }, 2024-08-20T22:07:49.6009484Z { 2024-08-20T22:07:49.6010014Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6010681Z "size": 907, 2024-08-20T22:07:49.6011329Z "digest": "sha256:768c31dca016b80bbdc973c8ee818034bdbd17b7ea9bdadbf0ec45ea51005720" 2024-08-20T22:07:49.6012196Z }, 2024-08-20T22:07:49.6012555Z { 2024-08-20T22:07:49.6013125Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6013728Z "size": 701, 2024-08-20T22:07:49.6014444Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:07:49.6015194Z }, 2024-08-20T22:07:49.6015517Z { 2024-08-20T22:07:49.6016051Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6016735Z "size": 134, 2024-08-20T22:07:49.6017379Z "digest": "sha256:8c32d0ecebbe2054c96a3a5c1f3d6c8b87c2e826686b369e5242cd55d0f01c2d" 2024-08-20T22:07:49.6018052Z }, 2024-08-20T22:07:49.6018444Z { 2024-08-20T22:07:49.6018991Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6019768Z "size": 32, 2024-08-20T22:07:49.6020370Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:07:49.6021096Z }, 2024-08-20T22:07:49.6021561Z { 2024-08-20T22:07:49.6022108Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6022710Z "size": 158, 2024-08-20T22:07:49.6023403Z "digest": "sha256:78ee77eddef23f20afc78e2924f4467041cadb754634ee3df431c68211cf676d" 2024-08-20T22:07:49.6024135Z }, 2024-08-20T22:07:49.6024436Z { 2024-08-20T22:07:49.6025008Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6025810Z "size": 1841, 2024-08-20T22:07:49.6026441Z "digest": "sha256:37513e9b43d7123e7ddee09100f62a72b57af7b3833eaecd515e247e2f63e13f" 2024-08-20T22:07:49.6027182Z }, 2024-08-20T22:07:49.6027516Z { 2024-08-20T22:07:49.6028110Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6028823Z "size": 7529776, 2024-08-20T22:07:49.6029431Z "digest": "sha256:a7a5392e2bf6b73d9be292cc22a99ad4637636c15733beb7afbe6403b5ba21a8" 2024-08-20T22:07:49.6030133Z }, 2024-08-20T22:07:49.6030539Z { 2024-08-20T22:07:49.6031027Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6031687Z "size": 163, 2024-08-20T22:07:49.6032350Z "digest": "sha256:b10de02255c021b2366b85bf12ddf593aac89f88964201c455b71a145bc5ca2e" 2024-08-20T22:07:49.6033050Z }, 2024-08-20T22:07:49.6033374Z { 2024-08-20T22:07:49.6033944Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6034598Z "size": 7943, 2024-08-20T22:07:49.6035324Z "digest": "sha256:e56eeff2db93ffe5002a2bb764795b7333321c6802cc28bef0801a619fe6a07e" 2024-08-20T22:07:49.6036043Z }, 2024-08-20T22:07:49.6036378Z { 2024-08-20T22:07:49.6036915Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6037611Z "size": 8068, 2024-08-20T22:07:49.6038226Z "digest": "sha256:70ac3e38d0fd1f0f46ad710c3fc485fd01406606054e8cf84a63af1570bc9f3a" 2024-08-20T22:07:49.6038934Z }, 2024-08-20T22:07:49.6039328Z { 2024-08-20T22:07:49.6039831Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6040466Z "size": 302, 2024-08-20T22:07:49.6041135Z "digest": "sha256:eb09cbf3e4168749cc8605e3188ed35070f6037d13930e443c38d8941734488f" 2024-08-20T22:07:49.6041930Z }, 2024-08-20T22:07:49.6042232Z { 2024-08-20T22:07:49.6042830Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6043489Z "size": 7629638, 2024-08-20T22:07:49.6044171Z "digest": "sha256:ee1273153675e6543b58f5be50321abee5e91f7351701d5d501a6837eaf676ef" 2024-08-20T22:07:49.6044853Z }, 2024-08-20T22:07:49.6045190Z { 2024-08-20T22:07:49.6045762Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6046383Z "size": 108, 2024-08-20T22:07:49.6047016Z "digest": "sha256:e22979f9e0eedfa16f98eb4fa26f30a50c237f5ffd2f6e5b898f2b908058496d" 2024-08-20T22:07:49.6047773Z }, 2024-08-20T22:07:49.6048134Z { 2024-08-20T22:07:49.6048686Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6049500Z "size": 54145661, 2024-08-20T22:07:49.6050165Z "digest": "sha256:6c1f0aabce7c84f9b0b9ab3b96325778dba1c1465b7463c5b4c6be1cf52b74db" 2024-08-20T22:07:49.6050881Z }, 2024-08-20T22:07:49.6051259Z { 2024-08-20T22:07:49.6051779Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6052414Z "size": 473, 2024-08-20T22:07:49.6053116Z "digest": "sha256:7ebe49c3a23e71b289ce012b48dad35791a2f2aa7fcee2326a64d763b0d14df8" 2024-08-20T22:07:49.6053798Z }, 2024-08-20T22:07:49.6054161Z { 2024-08-20T22:07:49.6054739Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6055427Z "size": 1374858766, 2024-08-20T22:07:49.6056105Z "digest": "sha256:5d115b29b5d53e5418762bc41ae975dd245bcffc3d058b8902c58146d505cb37" 2024-08-20T22:07:49.6056866Z }, 2024-08-20T22:07:49.6057205Z { 2024-08-20T22:07:49.6057707Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6058434Z "size": 106, 2024-08-20T22:07:49.6059087Z "digest": "sha256:7f2d7a04fbe65bbe1e895138e0cfeeef53ad926c890bab8e7e132ad1e655e48f" 2024-08-20T22:07:49.6059950Z }, 2024-08-20T22:07:49.6060308Z { 2024-08-20T22:07:49.6060843Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6061486Z "size": 559, 2024-08-20T22:07:49.6062224Z "digest": "sha256:d79fd7d526989e554455eda7d459f2c88444030f61710101e52ec8c7a49b7b79" 2024-08-20T22:07:49.6063055Z }, 2024-08-20T22:07:49.6063399Z { 2024-08-20T22:07:49.6063997Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6064596Z "size": 46248488, 2024-08-20T22:07:49.6065241Z "digest": "sha256:9752d6fbbcab4e7403f26777c5b65911320de6389c81b833a47ba45c4f3354b4" 2024-08-20T22:07:49.6066017Z }, 2024-08-20T22:07:49.6066358Z { 2024-08-20T22:07:49.6066845Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6067558Z "size": 111, 2024-08-20T22:07:49.6068195Z "digest": "sha256:83808b2eff4d71ed46db7c29f66fe918f328405c25b314225dfad01b4868d1f4" 2024-08-20T22:07:49.6069026Z }, 2024-08-20T22:07:49.6069417Z { 2024-08-20T22:07:49.6069945Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6070641Z "size": 32, 2024-08-20T22:07:49.6071270Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:07:49.6071947Z }, 2024-08-20T22:07:49.6072367Z { 2024-08-20T22:07:49.6072938Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6073560Z "size": 32, 2024-08-20T22:07:49.6074274Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:07:49.6074980Z }, 2024-08-20T22:07:49.6075337Z { 2024-08-20T22:07:49.6075949Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6076590Z "size": 32, 2024-08-20T22:07:49.6077251Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:07:49.6270471Z }, 2024-08-20T22:07:49.6270846Z { 2024-08-20T22:07:49.6271307Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:07:49.6271870Z "size": 32, 2024-08-20T22:07:49.6272403Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:07:49.6273028Z } 2024-08-20T22:07:49.6273274Z ] 2024-08-20T22:07:49.6273540Z } 2024-08-20T22:07:49.6373959Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:07:49.6374447Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:07:49.6375004Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T22:07:49.6384165Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:49.6384669Z env: 2024-08-20T22:07:49.6384969Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:49.6386126Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:49.6387299Z ##[endgroup] 2024-08-20T22:07:49.6420228Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:49.6471384Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T22:07:49.6471960Z with: 2024-08-20T22:07:49.6473040Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:49.6474344Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:49.6474855Z env: 2024-08-20T22:07:49.6475134Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:49.6475479Z ##[endgroup] 2024-08-20T22:07:49.6506401Z ##[group]Run set -x 2024-08-20T22:07:49.6506744Z set -x 2024-08-20T22:07:49.6507050Z set +e 2024-08-20T22:07:49.6507349Z  2024-08-20T22:07:49.6507627Z login() { 2024-08-20T22:07:49.6508292Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:07:49.6509011Z } 2024-08-20T22:07:49.6509295Z  2024-08-20T22:07:49.6509602Z retry () { 2024-08-20T22:07:49.6510001Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:07:49.6510445Z } 2024-08-20T22:07:49.6510904Z  2024-08-20T22:07:49.6511235Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:07:49.6511656Z  2024-08-20T22:07:49.6511943Z set -e 2024-08-20T22:07:49.6512428Z # ignore output since only exit code is used for conditional 2024-08-20T22:07:49.6513151Z # only pull docker image if it's not available locally 2024-08-20T22:07:49.6513922Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T22:07:49.6514621Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T22:07:49.6515072Z fi 2024-08-20T22:07:49.6523286Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:07:49.6523785Z env: 2024-08-20T22:07:49.6524066Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:07:49.6525215Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:49.6526525Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:49.6527043Z ##[endgroup] 2024-08-20T22:07:49.6554578Z + set +e 2024-08-20T22:07:49.6555231Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:49.6555888Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:49.6558241Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:07:49.6560621Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:07:50.1897318Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:07:50.1898153Z Configure a credential helper to remove this warning. See 2024-08-20T22:07:50.1898977Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:07:50.1899734Z 2024-08-20T22:07:50.1899893Z Login Succeeded 2024-08-20T22:07:50.1921884Z + set -e 2024-08-20T22:07:50.1923367Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:50.2085227Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:50.2087383Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:07:50.4680666Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-08-20T22:07:50.4682750Z 7a2c55901189: Pulling fs layer 2024-08-20T22:07:50.4683320Z 224fe954d725: Pulling fs layer 2024-08-20T22:07:50.4684165Z 75722010b82e: Pulling fs layer 2024-08-20T22:07:50.4684601Z d527cbbb87e3: Pulling fs layer 2024-08-20T22:07:50.4685120Z b57676e46aee: Pulling fs layer 2024-08-20T22:07:50.4685659Z a8c1e85b5e14: Pulling fs layer 2024-08-20T22:07:50.4686219Z a41a8d1c11c8: Pulling fs layer 2024-08-20T22:07:50.4686765Z 0c1227890755: Pulling fs layer 2024-08-20T22:07:50.4687287Z d8d1234baab3: Pulling fs layer 2024-08-20T22:07:50.4687830Z 7ed32bc8e469: Pulling fs layer 2024-08-20T22:07:50.4688365Z ec1e7978c1fe: Pulling fs layer 2024-08-20T22:07:50.4688891Z ac23c5fcc1ef: Pulling fs layer 2024-08-20T22:07:50.4689407Z ca426ffc9d2b: Pulling fs layer 2024-08-20T22:07:50.4689906Z 4373aa743caf: Pulling fs layer 2024-08-20T22:07:50.4690392Z b57676e46aee: Waiting 2024-08-20T22:07:50.4690840Z 75b2abb16c68: Pulling fs layer 2024-08-20T22:07:50.4691290Z ef1e3ef1e85f: Pulling fs layer 2024-08-20T22:07:50.4691668Z 0bf61114d37b: Pulling fs layer 2024-08-20T22:07:50.4692044Z 2df849f72ba4: Pulling fs layer 2024-08-20T22:07:50.4692420Z 34c47d63c5de: Pulling fs layer 2024-08-20T22:07:50.4692779Z a8c1e85b5e14: Waiting 2024-08-20T22:07:50.4693524Z 1fef4bd19792: Pulling fs layer 2024-08-20T22:07:50.4694196Z 6efb96fc87ea: Pulling fs layer 2024-08-20T22:07:50.4694562Z a41a8d1c11c8: Waiting 2024-08-20T22:07:50.4694891Z b119c4fc6c58: Pulling fs layer 2024-08-20T22:07:50.4695267Z 33b639226eee: Pulling fs layer 2024-08-20T22:07:50.4695690Z d3547bd43146: Pulling fs layer 2024-08-20T22:07:50.4696103Z 0c1227890755: Waiting 2024-08-20T22:07:50.4696423Z d527cbbb87e3: Waiting 2024-08-20T22:07:50.4696748Z ef1e3ef1e85f: Waiting 2024-08-20T22:07:50.4697069Z d8d1234baab3: Waiting 2024-08-20T22:07:50.4697382Z 7ed32bc8e469: Waiting 2024-08-20T22:07:50.4697778Z 5645520256fa: Pulling fs layer 2024-08-20T22:07:50.4698148Z 0bf61114d37b: Waiting 2024-08-20T22:07:50.4698456Z ec1e7978c1fe: Waiting 2024-08-20T22:07:50.4698775Z ac23c5fcc1ef: Waiting 2024-08-20T22:07:50.4699127Z 829ed7d91e38: Pulling fs layer 2024-08-20T22:07:50.4699710Z 2df849f72ba4: Waiting 2024-08-20T22:07:50.4700047Z 385d0421d67c: Pulling fs layer 2024-08-20T22:07:50.4700429Z 7cbfce999d76: Pulling fs layer 2024-08-20T22:07:50.4700791Z b119c4fc6c58: Waiting 2024-08-20T22:07:50.4701131Z 6658e558ebe4: Pulling fs layer 2024-08-20T22:07:50.4701501Z 33444a5a7e01: Pulling fs layer 2024-08-20T22:07:50.4701873Z 7c47122f4e09: Pulling fs layer 2024-08-20T22:07:50.4702246Z 1811ff4d3556: Pulling fs layer 2024-08-20T22:07:50.4702612Z 7754ee068b33: Pulling fs layer 2024-08-20T22:07:50.4702984Z 0479fbac3361: Pulling fs layer 2024-08-20T22:07:50.4703469Z f1d6874dee9c: Pulling fs layer 2024-08-20T22:07:50.4703955Z 4373aa743caf: Waiting 2024-08-20T22:07:50.4704410Z afc429e87b59: Pulling fs layer 2024-08-20T22:07:50.4704919Z 019620471c25: Pulling fs layer 2024-08-20T22:07:50.4705479Z 27aeb1b2cd5e: Pulling fs layer 2024-08-20T22:07:50.4705874Z 385d0421d67c: Waiting 2024-08-20T22:07:50.4706181Z 829ed7d91e38: Waiting 2024-08-20T22:07:50.4706510Z 829c0e19f4b7: Pulling fs layer 2024-08-20T22:07:50.4706901Z 560cd6122a38: Pulling fs layer 2024-08-20T22:07:50.4707373Z 7754ee068b33: Waiting 2024-08-20T22:07:50.4707859Z 5a4eb19d7219: Pulling fs layer 2024-08-20T22:07:50.4708298Z 5645520256fa: Waiting 2024-08-20T22:07:50.4708627Z afc429e87b59: Waiting 2024-08-20T22:07:50.4708946Z 75b2abb16c68: Waiting 2024-08-20T22:07:50.4709365Z da6a838a66c7: Pulling fs layer 2024-08-20T22:07:50.4709915Z cbb79a705353: Pulling fs layer 2024-08-20T22:07:50.4710454Z 4d4fa6802ab3: Pulling fs layer 2024-08-20T22:07:50.4710972Z 019620471c25: Waiting 2024-08-20T22:07:50.4711429Z 0f085d4fa279: Pulling fs layer 2024-08-20T22:07:50.4711889Z d3547bd43146: Waiting 2024-08-20T22:07:50.4712201Z f1d6874dee9c: Waiting 2024-08-20T22:07:50.4712522Z 27aeb1b2cd5e: Waiting 2024-08-20T22:07:50.4712904Z 829c0e19f4b7: Waiting 2024-08-20T22:07:50.4713340Z 1fef4bd19792: Waiting 2024-08-20T22:07:50.4713818Z 4a0412373305: Pulling fs layer 2024-08-20T22:07:50.4714368Z 302c8c177944: Pulling fs layer 2024-08-20T22:07:50.4714865Z 02970347dff6: Pulling fs layer 2024-08-20T22:07:50.4715612Z 916c5e858016: Pulling fs layer 2024-08-20T22:07:50.4716142Z c42b3602a1fb: Pulling fs layer 2024-08-20T22:07:50.4716673Z 11fad972ef6c: Pulling fs layer 2024-08-20T22:07:50.4717205Z f957c1624481: Pulling fs layer 2024-08-20T22:07:50.4717714Z 768c31dca016: Pulling fs layer 2024-08-20T22:07:50.4718217Z da6a838a66c7: Waiting 2024-08-20T22:07:50.4718653Z 560cd6122a38: Waiting 2024-08-20T22:07:50.4719128Z 8c32d0ecebbe: Pulling fs layer 2024-08-20T22:07:50.4719641Z cbb79a705353: Waiting 2024-08-20T22:07:50.4720089Z 1811ff4d3556: Waiting 2024-08-20T22:07:50.4720550Z 4f4fb700ef54: Pulling fs layer 2024-08-20T22:07:50.4721057Z 78ee77eddef2: Pulling fs layer 2024-08-20T22:07:50.4721566Z 6658e558ebe4: Waiting 2024-08-20T22:07:50.4722020Z 37513e9b43d7: Pulling fs layer 2024-08-20T22:07:50.4722519Z 33444a5a7e01: Waiting 2024-08-20T22:07:50.4722963Z a7a5392e2bf6: Pulling fs layer 2024-08-20T22:07:50.4723484Z 7c47122f4e09: Waiting 2024-08-20T22:07:50.4723952Z b10de02255c0: Pulling fs layer 2024-08-20T22:07:50.4724477Z 0f085d4fa279: Waiting 2024-08-20T22:07:50.4724937Z e56eeff2db93: Pulling fs layer 2024-08-20T22:07:50.4725442Z 4a0412373305: Waiting 2024-08-20T22:07:50.4725897Z 70ac3e38d0fd: Pulling fs layer 2024-08-20T22:07:50.4726499Z 302c8c177944: Waiting 2024-08-20T22:07:50.4726956Z eb09cbf3e416: Pulling fs layer 2024-08-20T22:07:50.4727466Z 768c31dca016: Waiting 2024-08-20T22:07:50.4727931Z ee1273153675: Pulling fs layer 2024-08-20T22:07:50.4728440Z a7a5392e2bf6: Waiting 2024-08-20T22:07:50.4728916Z e22979f9e0ee: Pulling fs layer 2024-08-20T22:07:50.4729458Z 6c1f0aabce7c: Pulling fs layer 2024-08-20T22:07:50.4729955Z 02970347dff6: Waiting 2024-08-20T22:07:50.4730388Z 0479fbac3361: Waiting 2024-08-20T22:07:50.4730826Z 4f4fb700ef54: Waiting 2024-08-20T22:07:50.4731291Z 7ebe49c3a23e: Pulling fs layer 2024-08-20T22:07:50.4731794Z 6efb96fc87ea: Waiting 2024-08-20T22:07:50.4732248Z e56eeff2db93: Waiting 2024-08-20T22:07:50.4732721Z 5d115b29b5d5: Pulling fs layer 2024-08-20T22:07:50.4733237Z 5a4eb19d7219: Waiting 2024-08-20T22:07:50.4733701Z 7f2d7a04fbe6: Pulling fs layer 2024-08-20T22:07:50.4734211Z 6c1f0aabce7c: Waiting 2024-08-20T22:07:50.4734644Z 916c5e858016: Waiting 2024-08-20T22:07:50.4735082Z 70ac3e38d0fd: Waiting 2024-08-20T22:07:50.4735541Z d79fd7d52698: Pulling fs layer 2024-08-20T22:07:50.4736041Z ee1273153675: Waiting 2024-08-20T22:07:50.4736480Z 4d4fa6802ab3: Waiting 2024-08-20T22:07:50.4736943Z 9752d6fbbcab: Pulling fs layer 2024-08-20T22:07:50.4737487Z 83808b2eff4d: Pulling fs layer 2024-08-20T22:07:50.4738002Z 7ebe49c3a23e: Waiting 2024-08-20T22:07:50.4738446Z c42b3602a1fb: Waiting 2024-08-20T22:07:50.4738871Z 5d115b29b5d5: Waiting 2024-08-20T22:07:50.4739426Z f957c1624481: Waiting 2024-08-20T22:07:50.4739864Z b10de02255c0: Waiting 2024-08-20T22:07:50.4740289Z 78ee77eddef2: Waiting 2024-08-20T22:07:50.4740729Z 33b639226eee: Waiting 2024-08-20T22:07:50.4741174Z eb09cbf3e416: Waiting 2024-08-20T22:07:50.4741619Z e22979f9e0ee: Waiting 2024-08-20T22:07:50.5887358Z 224fe954d725: Verifying Checksum 2024-08-20T22:07:50.5888001Z 224fe954d725: Download complete 2024-08-20T22:07:50.6955761Z d527cbbb87e3: Verifying Checksum 2024-08-20T22:07:50.6956275Z d527cbbb87e3: Download complete 2024-08-20T22:07:50.7674378Z b57676e46aee: Download complete 2024-08-20T22:07:50.8111099Z 7a2c55901189: Verifying Checksum 2024-08-20T22:07:50.8111555Z 7a2c55901189: Download complete 2024-08-20T22:07:50.8999415Z a41a8d1c11c8: Download complete 2024-08-20T22:07:50.9972821Z 0c1227890755: Verifying Checksum 2024-08-20T22:07:50.9973443Z 0c1227890755: Download complete 2024-08-20T22:07:51.0665138Z d8d1234baab3: Verifying Checksum 2024-08-20T22:07:51.0665746Z d8d1234baab3: Download complete 2024-08-20T22:07:51.0885286Z 75722010b82e: Verifying Checksum 2024-08-20T22:07:51.0885718Z 75722010b82e: Download complete 2024-08-20T22:07:51.1547910Z ec1e7978c1fe: Download complete 2024-08-20T22:07:51.2237972Z ac23c5fcc1ef: Verifying Checksum 2024-08-20T22:07:51.2238590Z ac23c5fcc1ef: Download complete 2024-08-20T22:07:51.9729982Z 7a2c55901189: Pull complete 2024-08-20T22:07:52.2915192Z 224fe954d725: Pull complete 2024-08-20T22:07:53.2166160Z 75722010b82e: Pull complete 2024-08-20T22:07:53.2361983Z d527cbbb87e3: Pull complete 2024-08-20T22:07:53.2568516Z b57676e46aee: Pull complete 2024-08-20T22:07:53.7506629Z ca426ffc9d2b: Verifying Checksum 2024-08-20T22:07:53.7507230Z ca426ffc9d2b: Download complete 2024-08-20T22:07:53.8308520Z 4373aa743caf: Verifying Checksum 2024-08-20T22:07:53.8309166Z 4373aa743caf: Download complete 2024-08-20T22:07:53.9329475Z 75b2abb16c68: Verifying Checksum 2024-08-20T22:07:53.9330098Z 75b2abb16c68: Download complete 2024-08-20T22:07:54.0525618Z ef1e3ef1e85f: Verifying Checksum 2024-08-20T22:07:54.0526229Z ef1e3ef1e85f: Download complete 2024-08-20T22:07:55.0171329Z 0bf61114d37b: Verifying Checksum 2024-08-20T22:07:55.0171794Z 0bf61114d37b: Download complete 2024-08-20T22:07:55.1071380Z 2df849f72ba4: Download complete 2024-08-20T22:07:55.1737923Z 34c47d63c5de: Verifying Checksum 2024-08-20T22:07:55.1738454Z 34c47d63c5de: Download complete 2024-08-20T22:07:55.2497658Z 1fef4bd19792: Verifying Checksum 2024-08-20T22:07:55.2498161Z 1fef4bd19792: Download complete 2024-08-20T22:08:04.1665858Z a8c1e85b5e14: Verifying Checksum 2024-08-20T22:08:04.1667677Z a8c1e85b5e14: Download complete 2024-08-20T22:08:04.3010260Z b119c4fc6c58: Verifying Checksum 2024-08-20T22:08:04.3010716Z b119c4fc6c58: Download complete 2024-08-20T22:08:04.3778761Z 33b639226eee: Download complete 2024-08-20T22:08:04.4591851Z d3547bd43146: Verifying Checksum 2024-08-20T22:08:04.4592561Z d3547bd43146: Download complete 2024-08-20T22:08:04.5300784Z 5645520256fa: Verifying Checksum 2024-08-20T22:08:04.5301420Z 5645520256fa: Download complete 2024-08-20T22:08:04.5951819Z 829ed7d91e38: Verifying Checksum 2024-08-20T22:08:04.5952409Z 829ed7d91e38: Download complete 2024-08-20T22:08:05.8702588Z 385d0421d67c: Verifying Checksum 2024-08-20T22:08:05.8703335Z 385d0421d67c: Download complete 2024-08-20T22:08:05.9531661Z 7cbfce999d76: Download complete 2024-08-20T22:08:06.0143647Z 6658e558ebe4: Download complete 2024-08-20T22:08:06.0842347Z 33444a5a7e01: Download complete 2024-08-20T22:08:06.1562686Z 7c47122f4e09: Verifying Checksum 2024-08-20T22:08:06.1563368Z 7c47122f4e09: Download complete 2024-08-20T22:08:06.2325029Z 1811ff4d3556: Download complete 2024-08-20T22:08:12.2416611Z 7754ee068b33: Verifying Checksum 2024-08-20T22:08:12.2417117Z 7754ee068b33: Download complete 2024-08-20T22:08:12.3237713Z 0479fbac3361: Verifying Checksum 2024-08-20T22:08:12.3238310Z 0479fbac3361: Download complete 2024-08-20T22:08:12.3906351Z f1d6874dee9c: Download complete 2024-08-20T22:08:12.8056129Z afc429e87b59: Verifying Checksum 2024-08-20T22:08:12.8958689Z 019620471c25: Verifying Checksum 2024-08-20T22:08:12.8959155Z 019620471c25: Download complete 2024-08-20T22:08:12.9707062Z 27aeb1b2cd5e: Verifying Checksum 2024-08-20T22:08:12.9708135Z 27aeb1b2cd5e: Download complete 2024-08-20T22:08:13.2554238Z 829c0e19f4b7: Verifying Checksum 2024-08-20T22:08:13.2554832Z 829c0e19f4b7: Download complete 2024-08-20T22:08:13.3286965Z 560cd6122a38: Download complete 2024-08-20T22:08:13.4113748Z 5a4eb19d7219: Verifying Checksum 2024-08-20T22:08:13.4114372Z 5a4eb19d7219: Download complete 2024-08-20T22:08:13.4836317Z da6a838a66c7: Download complete 2024-08-20T22:08:16.4040334Z 7ed32bc8e469: Verifying Checksum 2024-08-20T22:08:16.4040823Z 7ed32bc8e469: Download complete 2024-08-20T22:08:16.4769781Z 4d4fa6802ab3: Verifying Checksum 2024-08-20T22:08:16.4770344Z 4d4fa6802ab3: Download complete 2024-08-20T22:08:16.5792244Z 0f085d4fa279: Download complete 2024-08-20T22:08:17.0674729Z 4a0412373305: Verifying Checksum 2024-08-20T22:08:17.0675333Z 4a0412373305: Download complete 2024-08-20T22:08:17.1920221Z 302c8c177944: Verifying Checksum 2024-08-20T22:08:17.1920707Z 302c8c177944: Download complete 2024-08-20T22:08:17.2645145Z 02970347dff6: Verifying Checksum 2024-08-20T22:08:17.2645590Z 02970347dff6: Download complete 2024-08-20T22:08:17.3331418Z 916c5e858016: Download complete 2024-08-20T22:08:17.3420184Z a8c1e85b5e14: Pull complete 2024-08-20T22:08:17.4046316Z c42b3602a1fb: Verifying Checksum 2024-08-20T22:08:17.4046929Z c42b3602a1fb: Download complete 2024-08-20T22:08:17.4952485Z a41a8d1c11c8: Pull complete 2024-08-20T22:08:17.6342431Z 0c1227890755: Pull complete 2024-08-20T22:08:17.8423986Z d8d1234baab3: Pull complete 2024-08-20T22:08:23.9478813Z 6efb96fc87ea: Verifying Checksum 2024-08-20T22:08:23.9479705Z 6efb96fc87ea: Download complete 2024-08-20T22:08:24.0349855Z f957c1624481: Verifying Checksum 2024-08-20T22:08:24.0350456Z f957c1624481: Download complete 2024-08-20T22:08:24.1039896Z 768c31dca016: Verifying Checksum 2024-08-20T22:08:24.1040431Z 768c31dca016: Download complete 2024-08-20T22:08:24.1732603Z 8c32d0ecebbe: Download complete 2024-08-20T22:08:24.1849055Z 4f4fb700ef54: Verifying Checksum 2024-08-20T22:08:24.1849636Z 4f4fb700ef54: Download complete 2024-08-20T22:08:24.2686877Z 78ee77eddef2: Verifying Checksum 2024-08-20T22:08:24.2687448Z 78ee77eddef2: Download complete 2024-08-20T22:08:24.3436745Z 37513e9b43d7: Verifying Checksum 2024-08-20T22:08:24.3437372Z 37513e9b43d7: Download complete 2024-08-20T22:08:24.4848405Z a7a5392e2bf6: Verifying Checksum 2024-08-20T22:08:24.4848968Z a7a5392e2bf6: Download complete 2024-08-20T22:08:24.5457769Z b10de02255c0: Verifying Checksum 2024-08-20T22:08:24.5458364Z b10de02255c0: Download complete 2024-08-20T22:08:24.6273820Z e56eeff2db93: Verifying Checksum 2024-08-20T22:08:24.6274433Z e56eeff2db93: Download complete 2024-08-20T22:08:24.7015033Z 70ac3e38d0fd: Download complete 2024-08-20T22:08:24.7577595Z eb09cbf3e416: Verifying Checksum 2024-08-20T22:08:24.7578190Z eb09cbf3e416: Download complete 2024-08-20T22:08:24.9161726Z ee1273153675: Verifying Checksum 2024-08-20T22:08:24.9162375Z ee1273153675: Download complete 2024-08-20T22:08:24.9796182Z e22979f9e0ee: Download complete 2024-08-20T22:08:25.6058563Z 6c1f0aabce7c: Verifying Checksum 2024-08-20T22:08:25.6059075Z 6c1f0aabce7c: Download complete 2024-08-20T22:08:25.6702454Z 7ebe49c3a23e: Download complete 2024-08-20T22:08:51.7202322Z 5d115b29b5d5: Verifying Checksum 2024-08-20T22:08:51.7202981Z 5d115b29b5d5: Download complete 2024-08-20T22:08:51.8025977Z 7f2d7a04fbe6: Download complete 2024-08-20T22:08:51.8847011Z d79fd7d52698: Verifying Checksum 2024-08-20T22:08:51.8847480Z d79fd7d52698: Download complete 2024-08-20T22:08:53.2706245Z 9752d6fbbcab: Verifying Checksum 2024-08-20T22:08:53.2706810Z 9752d6fbbcab: Download complete 2024-08-20T22:08:53.3608878Z 83808b2eff4d: Verifying Checksum 2024-08-20T22:08:53.3609321Z 83808b2eff4d: Download complete 2024-08-20T22:09:31.8334922Z cbb79a705353: Verifying Checksum 2024-08-20T22:09:31.8335410Z cbb79a705353: Download complete 2024-08-20T22:09:32.6332369Z 11fad972ef6c: Verifying Checksum 2024-08-20T22:09:32.6332850Z 11fad972ef6c: Download complete 2024-08-20T22:09:47.0403114Z 7ed32bc8e469: Pull complete 2024-08-20T22:09:47.2813702Z ec1e7978c1fe: Pull complete 2024-08-20T22:09:47.5143002Z ac23c5fcc1ef: Pull complete 2024-08-20T22:09:56.2121764Z ca426ffc9d2b: Pull complete 2024-08-20T22:09:56.4356495Z 4373aa743caf: Pull complete 2024-08-20T22:09:56.6684248Z 75b2abb16c68: Pull complete 2024-08-20T22:09:56.9021827Z ef1e3ef1e85f: Pull complete 2024-08-20T22:09:59.5316190Z 0bf61114d37b: Pull complete 2024-08-20T22:09:59.6212345Z 2df849f72ba4: Pull complete 2024-08-20T22:09:59.7164257Z 34c47d63c5de: Pull complete 2024-08-20T22:09:59.8333264Z 1fef4bd19792: Pull complete 2024-08-20T22:10:58.2460585Z 6efb96fc87ea: Pull complete 2024-08-20T22:10:58.4772133Z b119c4fc6c58: Pull complete 2024-08-20T22:10:58.6806306Z 33b639226eee: Pull complete 2024-08-20T22:10:58.7881912Z d3547bd43146: Pull complete 2024-08-20T22:10:58.9902892Z 5645520256fa: Pull complete 2024-08-20T22:10:59.2252083Z 829ed7d91e38: Pull complete 2024-08-20T22:11:02.8671304Z 385d0421d67c: Pull complete 2024-08-20T22:11:03.0574548Z 7cbfce999d76: Pull complete 2024-08-20T22:11:03.2858518Z 6658e558ebe4: Pull complete 2024-08-20T22:11:03.4693527Z 33444a5a7e01: Pull complete 2024-08-20T22:11:03.6127814Z 7c47122f4e09: Pull complete 2024-08-20T22:11:03.8281905Z 1811ff4d3556: Pull complete 2024-08-20T22:11:14.1523278Z 7754ee068b33: Pull complete 2024-08-20T22:11:14.3854681Z 0479fbac3361: Pull complete 2024-08-20T22:11:14.6104901Z f1d6874dee9c: Pull complete 2024-08-20T22:11:15.5537243Z afc429e87b59: Pull complete 2024-08-20T22:11:15.7781802Z 019620471c25: Pull complete 2024-08-20T22:11:15.9904022Z 27aeb1b2cd5e: Pull complete 2024-08-20T22:11:16.4517128Z 829c0e19f4b7: Pull complete 2024-08-20T22:11:16.6729641Z 560cd6122a38: Pull complete 2024-08-20T22:11:17.0423349Z 5a4eb19d7219: Pull complete 2024-08-20T22:11:17.2504287Z da6a838a66c7: Pull complete 2024-08-20T22:11:56.0692735Z cbb79a705353: Pull complete 2024-08-20T22:11:56.2983828Z 4d4fa6802ab3: Pull complete 2024-08-20T22:11:56.5336646Z 0f085d4fa279: Pull complete 2024-08-20T22:11:58.9966546Z 4a0412373305: Pull complete 2024-08-20T22:11:59.0078078Z 302c8c177944: Pull complete 2024-08-20T22:11:59.0198147Z 02970347dff6: Pull complete 2024-08-20T22:11:59.0408512Z 916c5e858016: Pull complete 2024-08-20T22:11:59.0519581Z c42b3602a1fb: Pull complete 2024-08-20T22:13:21.8689222Z 11fad972ef6c: Pull complete 2024-08-20T22:13:22.0960302Z f957c1624481: Pull complete 2024-08-20T22:13:22.3254415Z 768c31dca016: Pull complete 2024-08-20T22:13:22.7355902Z 8c32d0ecebbe: Pull complete 2024-08-20T22:13:22.9518735Z 4f4fb700ef54: Pull complete 2024-08-20T22:13:23.1772567Z 78ee77eddef2: Pull complete 2024-08-20T22:13:23.3982454Z 37513e9b43d7: Pull complete 2024-08-20T22:13:23.7786729Z a7a5392e2bf6: Pull complete 2024-08-20T22:13:24.0012549Z b10de02255c0: Pull complete 2024-08-20T22:13:24.2352370Z e56eeff2db93: Pull complete 2024-08-20T22:13:24.3701636Z 70ac3e38d0fd: Pull complete 2024-08-20T22:13:24.5965931Z eb09cbf3e416: Pull complete 2024-08-20T22:13:26.0167638Z ee1273153675: Pull complete 2024-08-20T22:13:26.2410229Z e22979f9e0ee: Pull complete 2024-08-20T22:13:28.4873727Z 6c1f0aabce7c: Pull complete 2024-08-20T22:13:28.7141937Z 7ebe49c3a23e: Pull complete 2024-08-20T22:13:42.1495479Z 5d115b29b5d5: Pull complete 2024-08-20T22:13:42.3721733Z 7f2d7a04fbe6: Pull complete 2024-08-20T22:13:42.5993450Z d79fd7d52698: Pull complete 2024-08-20T22:13:43.4469810Z 9752d6fbbcab: Pull complete 2024-08-20T22:13:43.6152652Z 83808b2eff4d: Pull complete 2024-08-20T22:13:44.4892548Z Digest: sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T22:13:44.5352150Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:44.5557400Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:13:44.5649037Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:13:44.5650059Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:13:44.5660494Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:13:44.5660988Z env: 2024-08-20T22:13:44.5661265Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:44.5661613Z ##[endgroup] 2024-08-20T22:13:44.5862726Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-20T22:13:44.5863284Z with: 2024-08-20T22:13:44.5863578Z driver-version: 550.54.15 2024-08-20T22:13:44.5863928Z env: 2024-08-20T22:13:44.5864203Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:44.5864555Z ##[endgroup] 2024-08-20T22:13:44.5957948Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:13:44.5958512Z with: 2024-08-20T22:13:44.5958797Z timeout_minutes: 10 2024-08-20T22:13:44.5959122Z max_attempts: 3 2024-08-20T22:13:44.5991959Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-20T22:13:44.6025870Z retry_wait_seconds: 10 2024-08-20T22:13:44.6026234Z polling_interval_seconds: 1 2024-08-20T22:13:44.6026601Z warning_on_retry: true 2024-08-20T22:13:44.6026952Z continue_on_error: false 2024-08-20T22:13:44.6027287Z env: 2024-08-20T22:13:44.6027557Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:13:44.6027896Z DRIVER_VERSION: 550.54.15 2024-08-20T22:13:44.6028247Z ##[endgroup] 2024-08-20T22:13:44.6866259Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-20T22:13:44.6867065Z + pre_install_nvidia_driver_amzn2 2024-08-20T22:13:44.6870021Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-20T22:13:45.1957098Z No match for argument: nvidia-driver-latest-dkms 2024-08-20T22:13:45.1957755Z No packages marked for removal. 2024-08-20T22:13:45.2020467Z Dependencies resolved. 2024-08-20T22:13:45.2030423Z Nothing to do. 2024-08-20T22:13:45.2031092Z Complete! 2024-08-20T22:13:45.2798548Z + install_nvidia_driver_common 2024-08-20T22:13:45.2806812Z + echo 'Before installing NVIDIA driver' 2024-08-20T22:13:45.2807465Z + lspci 2024-08-20T22:13:45.2807824Z Before installing NVIDIA driver 2024-08-20T22:13:45.3644251Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:13:45.3645158Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:13:45.3646117Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:13:45.3646912Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:13:45.3647716Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:13:45.3648517Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:13:45.3649256Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:13:45.3650058Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:13:45.3650646Z + lsmod 2024-08-20T22:13:45.3687098Z Module Size Used by 2024-08-20T22:13:45.3687692Z xt_conntrack 16384 1 2024-08-20T22:13:45.3688554Z nft_chain_nat 16384 3 2024-08-20T22:13:45.3689618Z xt_MASQUERADE 20480 1 2024-08-20T22:13:45.3690795Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:13:45.3692089Z nf_conntrack_netlink 57344 0 2024-08-20T22:13:45.3694001Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:13:45.3695700Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:13:45.3696911Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:13:45.3697900Z xfrm_user 57344 1 2024-08-20T22:13:45.3698537Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:13:45.3699115Z xt_addrtype 16384 2 2024-08-20T22:13:45.3699811Z nft_compat 20480 4 2024-08-20T22:13:45.3700316Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:13:45.3700960Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:13:45.3701507Z br_netfilter 36864 0 2024-08-20T22:13:45.3701916Z bridge 307200 1 br_netfilter 2024-08-20T22:13:45.3702358Z stp 16384 1 bridge 2024-08-20T22:13:45.3702771Z llc 16384 2 bridge,stp 2024-08-20T22:13:45.3703185Z overlay 167936 0 2024-08-20T22:13:45.3703553Z tls 114688 0 2024-08-20T22:13:45.3703916Z nls_ascii 16384 1 2024-08-20T22:13:45.3704274Z nls_cp437 20480 1 2024-08-20T22:13:45.3704639Z vfat 24576 1 2024-08-20T22:13:45.3705011Z fat 86016 1 vfat 2024-08-20T22:13:45.3705397Z sunrpc 692224 1 2024-08-20T22:13:45.3705759Z ena 167936 0 2024-08-20T22:13:45.3706127Z ghash_clmulni_intel 16384 0 2024-08-20T22:13:45.3706511Z ptp 36864 1 ena 2024-08-20T22:13:45.3706900Z pps_core 24576 1 ptp 2024-08-20T22:13:45.3707295Z aesni_intel 393216 0 2024-08-20T22:13:45.3708052Z i8042 45056 0 2024-08-20T22:13:45.3708631Z serio 28672 3 i8042 2024-08-20T22:13:45.3709175Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:13:45.3709807Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:13:45.3710407Z button 24576 0 2024-08-20T22:13:45.3710782Z sch_fq_codel 20480 17 2024-08-20T22:13:45.3711151Z dm_mod 188416 0 2024-08-20T22:13:45.3711515Z fuse 163840 1 2024-08-20T22:13:45.3711879Z configfs 57344 1 2024-08-20T22:13:45.3712240Z loop 36864 0 2024-08-20T22:13:45.3712816Z dax 45056 1 dm_mod 2024-08-20T22:13:45.3713213Z dmi_sysfs 20480 0 2024-08-20T22:13:45.3713582Z crc32_pclmul 16384 0 2024-08-20T22:13:45.3713943Z crc32c_intel 24576 0 2024-08-20T22:13:45.3714308Z efivarfs 24576 1 2024-08-20T22:13:45.3714682Z + modinfo nvidia 2024-08-20T22:13:45.3715382Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:13:45.3716073Z alias: char-major-195-* 2024-08-20T22:13:45.3716453Z version: 550.54.15 2024-08-20T22:13:45.3716800Z supported: external 2024-08-20T22:13:45.3717144Z license: NVIDIA 2024-08-20T22:13:45.3717526Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:13:45.3718051Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:13:45.3718509Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:13:45.3718974Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:13:45.3719470Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:13:45.3719963Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:13:45.3720456Z depends: i2c-core,drm 2024-08-20T22:13:45.3720816Z retpoline: Y 2024-08-20T22:13:45.3721118Z name: nvidia 2024-08-20T22:13:45.3721902Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:13:45.3722622Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:13:45.3723261Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:13:45.3723857Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:13:45.3724302Z parm: NVreg_RmLogonRC:int 2024-08-20T22:13:45.3724734Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:13:45.3725187Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:13:45.3725626Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:13:45.3726062Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:13:45.3726590Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:13:45.3727146Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:13:45.3727626Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:13:45.3728109Z parm: NVreg_EnableMSI:int 2024-08-20T22:13:45.3728529Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:13:45.3728990Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:13:45.3729521Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:13:45.3730101Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:13:45.3730657Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:13:45.3731255Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:13:45.3731849Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:13:45.3732459Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:13:45.3733049Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:13:45.3733533Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:13:45.3734065Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:13:45.3734611Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:13:45.3735099Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:13:45.3736779Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:13:45.3737265Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:13:45.3737738Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:13:45.3738182Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:13:45.3738683Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:13:45.3739206Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:13:45.3739788Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:13:45.3740268Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:13:45.3740762Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:13:45.3741246Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:13:45.3741742Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:13:45.3742212Z parm: NVreg_RmMsg:charp 2024-08-20T22:13:45.3742621Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:13:45.3743086Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:13:45.3743563Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:13:45.3744014Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:13:45.3744491Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:13:45.3744968Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:13:45.3745410Z parm: rm_firmware_active:charp 2024-08-20T22:13:45.3745824Z + HAS_NVIDIA_DRIVER=0 2024-08-20T22:13:45.3746212Z ++ command -v nvidia-smi 2024-08-20T22:13:45.3746617Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-20T22:13:45.3746979Z + set +e 2024-08-20T22:13:45.3747499Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-20T22:13:47.6245984Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-20T22:13:47.6246598Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:13:47.6247297Z + '[' 0 -ne 0 ']' 2024-08-20T22:13:47.6247664Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-20T22:13:47.6248043Z + HAS_NVIDIA_DRIVER=1 2024-08-20T22:13:47.6249279Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-20T22:13:47.6250167Z + set -e 2024-08-20T22:13:47.6250480Z + '[' 1 -eq 0 ']' 2024-08-20T22:13:47.6251063Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-20T22:13:47.6251763Z + post_install_nvidia_driver_common 2024-08-20T22:13:47.6254610Z + sudo modprobe nvidia 2024-08-20T22:13:47.7788468Z + echo 'After installing NVIDIA driver' 2024-08-20T22:13:47.7788926Z + lspci 2024-08-20T22:13:47.7789229Z After installing NVIDIA driver 2024-08-20T22:13:47.7895795Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:13:47.7896808Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:13:47.7897784Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:13:47.7898840Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:13:47.7899863Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:13:47.7900731Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:13:47.7901732Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:13:47.7902829Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:13:47.7903442Z + lsmod 2024-08-20T22:13:47.7927274Z Module Size Used by 2024-08-20T22:13:47.7927824Z nvidia_uvm 4706304 0 2024-08-20T22:13:47.7928363Z nvidia 54071296 1 nvidia_uvm 2024-08-20T22:13:47.7928922Z drm 602112 1 nvidia 2024-08-20T22:13:47.7929506Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:13:47.7930058Z backlight 24576 1 drm 2024-08-20T22:13:47.7930476Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:13:47.7930898Z xt_conntrack 16384 1 2024-08-20T22:13:47.7931276Z nft_chain_nat 16384 3 2024-08-20T22:13:47.7931647Z xt_MASQUERADE 20480 1 2024-08-20T22:13:47.7932367Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:13:47.7932849Z nf_conntrack_netlink 57344 0 2024-08-20T22:13:47.7933420Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:13:47.7934052Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:13:47.7934538Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:13:47.7935118Z xfrm_user 57344 1 2024-08-20T22:13:47.7935629Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:13:47.7936135Z xt_addrtype 16384 2 2024-08-20T22:13:47.7936506Z nft_compat 20480 4 2024-08-20T22:13:47.7936951Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:13:47.7937561Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:13:47.7938102Z br_netfilter 36864 0 2024-08-20T22:13:47.7938508Z bridge 307200 1 br_netfilter 2024-08-20T22:13:47.7938945Z stp 16384 1 bridge 2024-08-20T22:13:47.7939374Z llc 16384 2 bridge,stp 2024-08-20T22:13:47.7939934Z overlay 167936 0 2024-08-20T22:13:47.7940292Z tls 114688 0 2024-08-20T22:13:47.7940658Z nls_ascii 16384 1 2024-08-20T22:13:47.7941026Z nls_cp437 20480 1 2024-08-20T22:13:47.7941388Z vfat 24576 1 2024-08-20T22:13:47.7941748Z fat 86016 1 vfat 2024-08-20T22:13:47.7942136Z sunrpc 692224 1 2024-08-20T22:13:47.7942501Z ena 167936 0 2024-08-20T22:13:47.7942871Z ghash_clmulni_intel 16384 0 2024-08-20T22:13:47.7943241Z ptp 36864 1 ena 2024-08-20T22:13:47.7943644Z pps_core 24576 1 ptp 2024-08-20T22:13:47.7944170Z aesni_intel 393216 0 2024-08-20T22:13:47.7944659Z i8042 45056 0 2024-08-20T22:13:47.7945140Z serio 28672 3 i8042 2024-08-20T22:13:47.7945965Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:13:47.7946683Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:13:47.7947327Z button 24576 0 2024-08-20T22:13:47.7947804Z sch_fq_codel 20480 17 2024-08-20T22:13:47.7948296Z dm_mod 188416 0 2024-08-20T22:13:47.7948747Z fuse 163840 1 2024-08-20T22:13:47.7949184Z configfs 57344 1 2024-08-20T22:13:47.7949690Z loop 36864 0 2024-08-20T22:13:47.7950191Z dax 45056 1 dm_mod 2024-08-20T22:13:47.7950725Z dmi_sysfs 20480 0 2024-08-20T22:13:47.7951207Z crc32_pclmul 16384 0 2024-08-20T22:13:47.7951705Z crc32c_intel 24576 0 2024-08-20T22:13:47.7952085Z efivarfs 24576 1 2024-08-20T22:13:47.7952454Z + modinfo nvidia 2024-08-20T22:13:47.7953124Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:13:47.7953834Z alias: char-major-195-* 2024-08-20T22:13:47.7954225Z version: 550.54.15 2024-08-20T22:13:47.7954574Z supported: external 2024-08-20T22:13:47.7954912Z license: NVIDIA 2024-08-20T22:13:47.7955292Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:13:47.7955774Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:13:47.7956232Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:13:47.7956690Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:13:47.7957191Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:13:47.7957684Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:13:47.7958166Z depends: i2c-core,drm 2024-08-20T22:13:47.7958533Z retpoline: Y 2024-08-20T22:13:47.7958841Z name: nvidia 2024-08-20T22:13:47.7959446Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:13:47.7960144Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:13:47.7960794Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:13:47.7961536Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:13:47.7961982Z parm: NVreg_RmLogonRC:int 2024-08-20T22:13:47.7962407Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:13:47.7962862Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:13:47.7963299Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:13:47.7963737Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:13:47.7964258Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:13:47.7964816Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:13:47.7965298Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:13:47.7965725Z parm: NVreg_EnableMSI:int 2024-08-20T22:13:47.7966128Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:13:47.7966580Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:13:47.7967100Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:13:47.7967677Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:13:47.7968227Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:13:47.7968827Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:13:47.7969415Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:13:47.7970010Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:13:47.7970593Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:13:47.7971074Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:13:47.7971603Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:13:47.7972140Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:13:47.7972626Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:13:47.7973089Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:13:47.7973569Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:13:47.7974029Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:13:47.7974584Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:13:47.7975088Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:13:47.7975614Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:13:47.7976096Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:13:47.7976578Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:13:47.7977070Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:13:47.7977554Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:13:47.7978044Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:13:47.7978568Z parm: NVreg_RmMsg:charp 2024-08-20T22:13:47.7978975Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:13:47.7979437Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:13:47.7980028Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:13:47.7980477Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:13:47.7980957Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:13:47.7981436Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:13:47.7981877Z parm: rm_firmware_active:charp 2024-08-20T22:13:47.7982272Z + set +e 2024-08-20T22:13:47.7982579Z + nvidia-smi 2024-08-20T22:13:49.3664336Z Tue Aug 20 22:13:49 2024 2024-08-20T22:13:49.3665232Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:13:49.3666182Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:13:49.3667069Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:13:49.3667961Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:13:49.3668943Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:13:49.3669700Z | | | MIG M. | 2024-08-20T22:13:49.3670519Z |=========================================+========================+======================| 2024-08-20T22:13:49.3765404Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:13:49.3766150Z | 0% 29C P0 60W / 300W | 0MiB / 23028MiB | 5% Default | 2024-08-20T22:13:49.3766799Z | | | N/A | 2024-08-20T22:13:49.3767505Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:13:49.3768910Z 2024-08-20T22:13:49.3769822Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:13:49.3770473Z | Processes: | 2024-08-20T22:13:49.3771186Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:13:49.3771888Z | ID ID Usage | 2024-08-20T22:13:49.3772458Z |=========================================================================================| 2024-08-20T22:13:49.3773116Z | No running processes found | 2024-08-20T22:13:49.3773877Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:13:49.9968607Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-20T22:13:51.5695164Z NVIDIA A10G 2024-08-20T22:13:52.0066917Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:13:52.0067581Z + '[' 0 -eq 0 ']' 2024-08-20T22:13:52.0068301Z + echo 'INFO: Ignoring allowed status 0' 2024-08-20T22:13:52.0068877Z + set -e 2024-08-20T22:13:52.0069278Z INFO: Ignoring allowed status 0 2024-08-20T22:13:52.0078760Z == Installing nvidia container toolkit for amzn2023 == 2024-08-20T22:13:52.0083097Z + sudo yum install -y yum-utils 2024-08-20T22:13:52.4388786Z Last metadata expiration check: 0:09:38 ago on Tue Aug 20 22:04:14 2024. 2024-08-20T22:13:52.4777787Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-20T22:13:52.5080084Z Dependencies resolved. 2024-08-20T22:13:52.5205846Z Nothing to do. 2024-08-20T22:13:52.5206241Z Complete! 2024-08-20T22:13:52.6160919Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-20T22:13:52.6162150Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:13:52.6163713Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:13:52.8892021Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:13:52.9564597Z + sudo yum install -y nvidia-docker2 2024-08-20T22:13:53.5034442Z nvidia-container-toolkit 10 kB/s | 833 B 00:00 2024-08-20T22:13:53.5258667Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-20T22:13:53.5559223Z Dependencies resolved. 2024-08-20T22:13:53.5684607Z Nothing to do. 2024-08-20T22:13:53.5685404Z Complete! 2024-08-20T22:13:53.6605898Z + sudo systemctl restart docker 2024-08-20T22:14:47.2463181Z Tue Aug 20 22:14:47 2024 2024-08-20T22:14:47.2464295Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:47.2465305Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:14:47.2466254Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:14:47.2467209Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:14:47.2468265Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:14:47.2469100Z | | | MIG M. | 2024-08-20T22:14:47.2470147Z |=========================================+========================+======================| 2024-08-20T22:14:47.2593303Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:14:47.2594187Z | 0% 29C P0 59W / 300W | 0MiB / 23028MiB | 5% Default | 2024-08-20T22:14:47.2594990Z | | | N/A | 2024-08-20T22:14:47.2595855Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:14:47.2596556Z 2024-08-20T22:14:47.2597334Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:47.2598097Z | Processes: | 2024-08-20T22:14:47.2598940Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:14:47.2599753Z | ID ID Usage | 2024-08-20T22:14:47.2600421Z |=========================================================================================| 2024-08-20T22:14:47.2601188Z | No running processes found | 2024-08-20T22:14:47.2602096Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:47.7235303Z Command completed after 1 attempt(s). 2024-08-20T22:14:47.7318342Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:14:47.7319121Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:14:47.7319819Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T22:14:47.7320476Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:14:47.7335045Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:14:47.7335556Z env: 2024-08-20T22:14:47.7335850Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:47.7336300Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:14:47.7336788Z ##[endgroup] 2024-08-20T22:14:48.0246392Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:14:48.4431393Z Collecting psutil==5.9.1 2024-08-20T22:14:48.4789894Z Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB) 2024-08-20T22:14:48.5309790Z Collecting nvidia-ml-py==11.525.84 2024-08-20T22:14:48.5346229Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2024-08-20T22:14:48.6289007Z Installing collected packages: psutil, nvidia-ml-py 2024-08-20T22:14:48.7852687Z Successfully installed nvidia-ml-py-11.525.84 psutil-5.9.1 2024-08-20T22:14:48.9822549Z Prepare all required actions 2024-08-20T22:14:48.9823067Z Getting action download info 2024-08-20T22:14:49.0992596Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T22:14:49.3418545Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T22:14:49.4712742Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T22:14:49.4713233Z with: 2024-08-20T22:14:49.4713584Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:14:49.4714051Z s3-bucket: gha-artifacts 2024-08-20T22:14:49.4714393Z env: 2024-08-20T22:14:49.4714676Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:49.4715129Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:14:49.4715618Z ##[endgroup] 2024-08-20T22:14:49.4749327Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:14:49.4749768Z with: 2024-08-20T22:14:49.4750128Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:14:49.4750748Z s3-bucket: gha-artifacts 2024-08-20T22:14:49.4751092Z region: us-east-1 2024-08-20T22:14:49.4751388Z env: 2024-08-20T22:14:49.4751656Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:49.4752098Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:14:49.4752567Z ##[endgroup] 2024-08-20T22:14:49.9505266Z (node:52479) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:14:49.9505975Z 2024-08-20T22:14:49.9506246Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:14:49.9506982Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:14:49.9507860Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:14:50.0250398Z Found 1 objects with prefix pytorch/pytorch/10479310721/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-08-20T22:14:50.0251507Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:15:15.9545866Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:15:15.9552593Z Artifact download has finished successfully 2024-08-20T22:15:15.9905371Z ##[group]Run unzip -o artifacts.zip 2024-08-20T22:15:15.9905808Z unzip -o artifacts.zip 2024-08-20T22:15:15.9914689Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:15.9915181Z env: 2024-08-20T22:15:15.9915458Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:15.9915913Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:15.9916395Z ##[endgroup] 2024-08-20T22:15:16.0041773Z Archive: artifacts.zip 2024-08-20T22:15:16.0043223Z creating: dist/ 2024-08-20T22:15:18.1365382Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:15:18.1366724Z creating: build/custom_test_artifacts/ 2024-08-20T22:15:18.1368058Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T22:15:18.1370057Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T22:15:18.1372047Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:18.1373841Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:18.1375023Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:18.1376153Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:18.1377237Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:18.1378214Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:18.1379636Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:18.1382709Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:18.1383855Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:18.1384927Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:18.1386272Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:18.1387819Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:18.1390253Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:18.1391424Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:18.1393418Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:18.1395177Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:18.1396498Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:18.1397526Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:18.1437224Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:18.1477055Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:18.1478704Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:18.1525498Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:18.1528317Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:18.1531187Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:18.1533375Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:18.1534794Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:18.1536179Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:18.1537580Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:18.1539094Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:18.1540457Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:18.1541735Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:18.1542983Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:18.1544208Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:18.1545452Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:18.1546652Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:18.1547862Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:18.1602516Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:18.1663094Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:18.1664281Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:18.1665280Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:18.1666153Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:18.1667059Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:18.1667987Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T22:15:18.1669018Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T22:15:18.1670192Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T22:15:18.1671475Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T22:15:18.1672531Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T22:15:18.1673617Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T22:15:18.1674715Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T22:15:18.1675807Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T22:15:18.1676896Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T22:15:18.1677959Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T22:15:18.1698812Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T22:15:18.1842623Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T22:15:18.1843632Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T22:15:18.1844730Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T22:15:18.1845952Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T22:15:18.1847135Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T22:15:18.1848242Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T22:15:18.1849387Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T22:15:18.1850540Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T22:15:18.1851704Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T22:15:18.1852846Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T22:15:18.1853969Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T22:15:18.1875401Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T22:15:18.1961083Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T22:15:18.1962438Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:18.1964619Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:18.1966887Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T22:15:18.1968753Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T22:15:18.1970547Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:18.1972326Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-20T22:15:18.1973197Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T22:15:18.1973995Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T22:15:18.1974826Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T22:15:18.2094055Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T22:15:18.2157548Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T22:15:18.2158311Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T22:15:18.2159031Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T22:15:18.2160016Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:18.2167101Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:18.2168040Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:18.2168996Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:18.2169991Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:18.2170963Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:18.2172499Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:18.2174876Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:18.2176010Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:18.2177129Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:18.2178600Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:18.2181055Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:18.2182749Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:18.2184005Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:18.2186304Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:18.2187800Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:18.2188903Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:18.2189924Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:18.2230980Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:18.2269620Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:18.2271055Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:18.2317011Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:18.2318417Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:18.2320122Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:18.2321620Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:18.2323024Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:18.2324402Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:18.2325784Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:18.2327158Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:18.2328510Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:18.2329945Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:18.2331185Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:18.2332400Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:18.2333639Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:18.2334831Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:18.2336046Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:18.2394083Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:18.2455801Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:18.2457000Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:18.2457998Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:18.2458960Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:18.2459854Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:18.2460808Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T22:15:18.2461880Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T22:15:18.2463080Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T22:15:18.2464242Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T22:15:18.2465328Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T22:15:18.2466444Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T22:15:18.2467560Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T22:15:18.2468683Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T22:15:18.2469796Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T22:15:18.2470904Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T22:15:18.2489802Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T22:15:18.2554788Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T22:15:18.2556004Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:18.2557091Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:18.2558065Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T22:15:18.2558976Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T22:15:18.2560000Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:18.2560910Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-20T22:15:18.2563995Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T22:15:18.2564872Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T22:15:18.2566733Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T22:15:18.2617711Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T22:15:18.2618479Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T22:15:18.2619343Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T22:15:18.2620232Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:18.2627071Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:18.2628072Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:18.2629074Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:18.2630131Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:18.2631173Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:18.2632352Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:18.2634191Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:18.2635267Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:18.2636318Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:18.2638113Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:18.2639851Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:18.2641745Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:18.2642971Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:18.2644690Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:18.2646082Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:18.2647205Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:18.2648292Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:18.2688229Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:18.2728323Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:18.2730040Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:18.2775613Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:18.2777089Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:18.2778598Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:18.2780245Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:18.2781721Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:18.2783169Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:18.2784777Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:18.2786232Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:18.2787646Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:18.2789002Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:18.2790308Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:18.2791583Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:18.2792922Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:18.2794463Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:18.2795740Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:18.2853530Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:18.2913642Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:18.2914910Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:18.2915982Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:18.2916919Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:18.2917908Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:18.2918933Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T22:15:18.2920084Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T22:15:18.2921346Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T22:15:18.2922566Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T22:15:18.2923734Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T22:15:18.2924916Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T22:15:18.2927285Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T22:15:18.2928513Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T22:15:18.2929711Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T22:15:18.2930890Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T22:15:18.2932284Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T22:15:18.3060472Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T22:15:18.3061642Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T22:15:18.3062894Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T22:15:18.3064409Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T22:15:18.3065684Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T22:15:18.3066890Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T22:15:18.3068135Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T22:15:18.3069386Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T22:15:18.3070635Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T22:15:18.3071882Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T22:15:18.3073175Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T22:15:18.3091381Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T22:15:18.3147315Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T22:15:18.3148624Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:18.3149772Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:18.3150825Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T22:15:18.3151801Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T22:15:18.3152830Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:18.3153817Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-20T22:15:18.3156563Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T22:15:18.3157516Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T22:15:18.3158499Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T22:15:18.3262373Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T22:15:18.3306716Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T22:15:18.3307345Z creating: build/lib/ 2024-08-20T22:15:18.3396129Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T22:15:18.3852280Z inflating: build/lib/libprotobuf.a 2024-08-20T22:15:18.3861561Z inflating: build/lib/libpthreadpool.a 2024-08-20T22:15:18.3869848Z inflating: build/lib/libcpuinfo.a 2024-08-20T22:15:18.3878055Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T22:15:18.3878681Z inflating: build/lib/libclog.a 2024-08-20T22:15:18.3897632Z inflating: build/lib/libnnpack.a 2024-08-20T22:15:18.3900463Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T22:15:18.3965342Z inflating: build/lib/libgtest.a 2024-08-20T22:15:18.4043131Z inflating: build/lib/libbenchmark.a 2024-08-20T22:15:18.4107038Z inflating: build/lib/libasmjit.a 2024-08-20T22:15:18.4114798Z inflating: build/lib/libittnotify.a 2024-08-20T22:15:18.4142807Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T22:15:18.4270042Z inflating: build/lib/libgloo.a 2024-08-20T22:15:18.4290801Z inflating: build/lib/libfmt.a 2024-08-20T22:15:18.4385760Z inflating: build/lib/libc10.so 2024-08-20T22:15:18.4387568Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-20T22:15:18.4389083Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T22:15:18.4893212Z inflating: build/lib/libprotoc.a 2024-08-20T22:15:18.4912722Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T22:15:18.4930683Z inflating: build/lib/libgmock.a 2024-08-20T22:15:18.4931233Z inflating: build/lib/libgtest_main.a 2024-08-20T22:15:18.4932245Z inflating: build/lib/libbenchmark_main.a 2024-08-20T22:15:18.5322950Z inflating: build/lib/libgloo_cuda.a 2024-08-20T22:15:19.5534253Z inflating: build/lib/libdnnl.a 2024-08-20T22:15:19.6108089Z inflating: build/lib/libtensorpipe.a 2024-08-20T22:15:19.6165567Z inflating: build/lib/libc10_cuda.so 2024-08-20T22:15:19.6166299Z inflating: build/lib/libgmock_main.a 2024-08-20T22:15:19.7430836Z inflating: build/lib/libfbgemm.a 2024-08-20T22:15:19.7686921Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-20T22:15:19.8192235Z inflating: build/lib/libkineto.a 2024-08-20T22:15:19.8378588Z inflating: build/lib/libXNNPACK.a 2024-08-20T22:15:19.8421439Z inflating: build/lib/libonnx_proto.a 2024-08-20T22:15:19.9114652Z inflating: build/lib/libonnx.a 2024-08-20T22:15:22.3654220Z inflating: build/lib/libtorch_cpu.so 2024-08-20T22:15:22.3659247Z inflating: build/lib/libunbox_lib.a 2024-08-20T22:15:22.3663822Z inflating: build/lib/libshm.so 2024-08-20T22:15:24.4387747Z inflating: build/lib/libtorch_cuda.so 2024-08-20T22:15:24.4388847Z inflating: build/lib/libtorch.so 2024-08-20T22:15:24.4391727Z inflating: build/lib/libc10d_cuda_test.so 2024-08-20T22:15:25.2599825Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-20T22:15:25.4602168Z inflating: build/lib/libtorch_python.so 2024-08-20T22:15:25.4674421Z inflating: build/lib/libtorchbind_test.so 2024-08-20T22:15:25.4695538Z inflating: build/lib/libjitbackend_test.so 2024-08-20T22:15:25.4720993Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T22:15:25.4747159Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T22:15:25.4782219Z inflating: build/lib/libnnapi_backend.so 2024-08-20T22:15:25.4783155Z creating: build/bin/ 2024-08-20T22:15:25.4833068Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T22:15:25.4885410Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T22:15:25.4936667Z inflating: build/bin/c10_Device_test 2024-08-20T22:15:25.4996398Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T22:15:25.5049882Z inflating: build/bin/c10_Scalar_test 2024-08-20T22:15:25.5102167Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T22:15:25.5153283Z inflating: build/bin/c10_SymInt_test 2024-08-20T22:15:25.5207973Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T22:15:25.5264914Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T22:15:25.5321466Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T22:15:25.5392743Z inflating: build/bin/c10_cow_test 2024-08-20T22:15:25.5446448Z inflating: build/bin/c10_Bitset_test 2024-08-20T22:15:25.5495384Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T22:15:25.5547909Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T22:15:25.5598941Z inflating: build/bin/c10_Half_test 2024-08-20T22:15:25.5655899Z inflating: build/bin/c10_LeftRight_test 2024-08-20T22:15:25.5710965Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T22:15:25.5761672Z inflating: build/bin/c10_Synchronized_test 2024-08-20T22:15:25.5817790Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T22:15:25.5869526Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T22:15:25.5921079Z inflating: build/bin/c10_TypeList_test 2024-08-20T22:15:25.5971913Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T22:15:25.6024654Z inflating: build/bin/c10_accumulate_test 2024-08-20T22:15:25.6080793Z inflating: build/bin/c10_bfloat16_test 2024-08-20T22:15:25.6131388Z inflating: build/bin/c10_bit_cast_test 2024-08-20T22:15:25.6188273Z inflating: build/bin/c10_complex_math_test 2024-08-20T22:15:25.6244446Z inflating: build/bin/c10_complex_test 2024-08-20T22:15:25.6297397Z inflating: build/bin/c10_exception_test 2024-08-20T22:15:25.6348157Z inflating: build/bin/c10_flags_test 2024-08-20T22:15:25.6398592Z inflating: build/bin/c10_generic_math_test 2024-08-20T22:15:25.6565816Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T22:15:25.6617469Z inflating: build/bin/c10_irange_test 2024-08-20T22:15:25.6670814Z inflating: build/bin/c10_lazy_test 2024-08-20T22:15:25.6728457Z inflating: build/bin/c10_logging_test 2024-08-20T22:15:25.6803964Z inflating: build/bin/c10_optional_test 2024-08-20T22:15:25.6866718Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T22:15:25.6921138Z inflating: build/bin/c10_registry_test 2024-08-20T22:15:25.7072951Z inflating: build/bin/c10_small_vector_test 2024-08-20T22:15:25.7125130Z inflating: build/bin/c10_ssize_test 2024-08-20T22:15:25.7177486Z inflating: build/bin/c10_string_util_test 2024-08-20T22:15:25.7228382Z inflating: build/bin/c10_tempfile_test 2024-08-20T22:15:25.7287337Z inflating: build/bin/c10_string_view_test 2024-08-20T22:15:25.7336397Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T22:15:25.7392847Z inflating: build/bin/c10_typeid_test 2024-08-20T22:15:25.7843692Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T22:15:25.8292444Z inflating: build/bin/protoc 2024-08-20T22:15:25.8345527Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-20T22:15:25.8399502Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-20T22:15:25.8452217Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-20T22:15:25.8504777Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-20T22:15:25.8560210Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-20T22:15:25.8612929Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-20T22:15:25.8662938Z inflating: build/bin/c10_cuda_CUDATest 2024-08-20T22:15:25.8715611Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-20T22:15:25.9047198Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T22:15:25.9393795Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T22:15:25.9756955Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T22:15:25.9810069Z inflating: build/bin/BackoffTest 2024-08-20T22:15:25.9863530Z inflating: build/bin/FileStoreTest 2024-08-20T22:15:25.9920145Z inflating: build/bin/TCPStoreTest 2024-08-20T22:15:25.9973228Z inflating: build/bin/HashStoreTest 2024-08-20T22:15:25.9987061Z inflating: build/bin/ProcessGroupMPITest 2024-08-20T22:15:26.0042159Z inflating: build/bin/test_edge_op_registration 2024-08-20T22:15:26.0046611Z inflating: build/bin/torch_shm_manager 2024-08-20T22:15:26.0049636Z inflating: build/bin/example_allreduce 2024-08-20T22:15:26.0105475Z inflating: build/bin/test_dist_autograd 2024-08-20T22:15:26.0174836Z inflating: build/bin/test_cpp_rpc 2024-08-20T22:15:26.0177203Z inflating: build/bin/parallel_benchmark 2024-08-20T22:15:26.0245018Z inflating: build/bin/test_mobile_nnc 2024-08-20T22:15:26.0253929Z inflating: build/bin/aot_model_compiler_test 2024-08-20T22:15:26.0603259Z inflating: build/bin/test_lazy 2024-08-20T22:15:26.1796406Z inflating: build/bin/test_api 2024-08-20T22:15:26.1870636Z inflating: build/bin/Dict_test 2024-08-20T22:15:26.1923262Z inflating: build/bin/Dimname_test 2024-08-20T22:15:26.1988021Z inflating: build/bin/MaybeOwned_test 2024-08-20T22:15:26.2045799Z inflating: build/bin/NamedTensor_test 2024-08-20T22:15:26.2105636Z inflating: build/bin/apply_utils_test 2024-08-20T22:15:26.2165732Z inflating: build/bin/atest 2024-08-20T22:15:26.2228759Z inflating: build/bin/basic 2024-08-20T22:15:26.2283884Z inflating: build/bin/broadcast_test 2024-08-20T22:15:26.2335559Z inflating: build/bin/cpu_allocator_test 2024-08-20T22:15:26.2394198Z inflating: build/bin/cpu_generator_test 2024-08-20T22:15:26.2448432Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T22:15:26.2541709Z inflating: build/bin/cpu_rng_test 2024-08-20T22:15:26.2591987Z inflating: build/bin/dispatch_key_set_test 2024-08-20T22:15:26.2645237Z inflating: build/bin/dlconvertor_test 2024-08-20T22:15:26.2703867Z inflating: build/bin/extension_backend_test 2024-08-20T22:15:26.2759687Z inflating: build/bin/half_test 2024-08-20T22:15:26.2855725Z inflating: build/bin/ivalue_test 2024-08-20T22:15:26.2905974Z inflating: build/bin/lazy_tensor_test 2024-08-20T22:15:26.2960638Z inflating: build/bin/math_kernel_test 2024-08-20T22:15:26.3015026Z inflating: build/bin/memory_format_test 2024-08-20T22:15:26.3068485Z inflating: build/bin/memory_overlapping_test 2024-08-20T22:15:26.3122468Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T22:15:26.3178204Z inflating: build/bin/native_test 2024-08-20T22:15:26.3229295Z inflating: build/bin/operator_name_test 2024-08-20T22:15:26.3280942Z inflating: build/bin/operators_test 2024-08-20T22:15:26.3333484Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T22:15:26.3400921Z inflating: build/bin/pow_test 2024-08-20T22:15:26.3458472Z inflating: build/bin/quantized_test 2024-08-20T22:15:26.3509039Z inflating: build/bin/reduce_ops_test 2024-08-20T22:15:26.3560774Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T22:15:26.3617252Z inflating: build/bin/scalar_tensor_test 2024-08-20T22:15:26.3678450Z inflating: build/bin/scalar_test 2024-08-20T22:15:26.3730789Z inflating: build/bin/StorageUtils_test 2024-08-20T22:15:26.3783713Z inflating: build/bin/stride_properties_test 2024-08-20T22:15:26.3863200Z inflating: build/bin/tensor_iterator_test 2024-08-20T22:15:26.3917788Z inflating: build/bin/test_parallel 2024-08-20T22:15:26.3920922Z inflating: build/bin/thread_init_test 2024-08-20T22:15:26.3977057Z inflating: build/bin/type_ptr_test 2024-08-20T22:15:26.4037948Z inflating: build/bin/type_test 2024-08-20T22:15:26.4090875Z inflating: build/bin/undefined_tensor_test 2024-08-20T22:15:26.4092330Z inflating: build/bin/verify_api_visibility 2024-08-20T22:15:26.4162451Z inflating: build/bin/legacy_vmap_test 2024-08-20T22:15:26.4214835Z inflating: build/bin/weakref_test 2024-08-20T22:15:26.4266325Z inflating: build/bin/wrapdim_test 2024-08-20T22:15:26.4318502Z inflating: build/bin/xla_tensor_test 2024-08-20T22:15:26.4378903Z inflating: build/bin/IListRef_test 2024-08-20T22:15:26.4486037Z inflating: build/bin/List_test 2024-08-20T22:15:26.4553188Z inflating: build/bin/KernelFunction_test 2024-08-20T22:15:26.4678940Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T22:15:26.4775847Z inflating: build/bin/kernel_function_test 2024-08-20T22:15:26.4905373Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T22:15:26.5010981Z inflating: build/bin/kernel_lambda_test 2024-08-20T22:15:26.5072497Z inflating: build/bin/kernel_stackbased_test 2024-08-20T22:15:26.5169802Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T22:15:26.5222240Z inflating: build/bin/CppSignature_test 2024-08-20T22:15:26.5271423Z inflating: build/bin/op_allowlist_test 2024-08-20T22:15:26.5327685Z inflating: build/bin/backend_fallback_test 2024-08-20T22:15:26.5632547Z inflating: build/bin/op_registration_test 2024-08-20T22:15:26.5700617Z inflating: build/bin/inline_container_test 2024-08-20T22:15:26.5754080Z inflating: build/bin/cuda_apply_test 2024-08-20T22:15:26.5806563Z inflating: build/bin/cuda_allocator_test 2024-08-20T22:15:26.5868471Z inflating: build/bin/cuda_atomic_ops_test 2024-08-20T22:15:26.5923705Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-20T22:15:26.5993804Z inflating: build/bin/cuda_complex_math_test 2024-08-20T22:15:26.6053533Z inflating: build/bin/cuda_complex_test 2024-08-20T22:15:26.6103565Z inflating: build/bin/cuda_device_test 2024-08-20T22:15:26.6161107Z inflating: build/bin/cuda_cub_test 2024-08-20T22:15:26.6212350Z inflating: build/bin/cuda_dlconvertor_test 2024-08-20T22:15:26.6278451Z inflating: build/bin/cuda_distributions_test 2024-08-20T22:15:26.6335735Z inflating: build/bin/cuda_generator_test 2024-08-20T22:15:26.6385320Z inflating: build/bin/cuda_half_test 2024-08-20T22:15:26.6437803Z inflating: build/bin/cuda_integer_divider_test 2024-08-20T22:15:26.6486842Z inflating: build/bin/cuda_optional_test 2024-08-20T22:15:26.6539479Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-20T22:15:26.6592589Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-20T22:15:26.6643252Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-20T22:15:26.6705907Z inflating: build/bin/cuda_stream_test 2024-08-20T22:15:26.6757595Z inflating: build/bin/cuda_cudnn_test 2024-08-20T22:15:26.6809536Z inflating: build/bin/cuda_vectorized_test 2024-08-20T22:15:26.6824160Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T22:15:26.6890239Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T22:15:26.6948582Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-20T22:15:26.7012732Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-20T22:15:26.7075249Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-20T22:15:26.7909559Z inflating: build/bin/test_tensorexpr 2024-08-20T22:15:26.8492573Z inflating: build/bin/test_jit 2024-08-20T22:15:26.8493714Z creating: .additional_ci_files/ 2024-08-20T22:15:26.8554458Z inflating: .additional_ci_files/test-times.json 2024-08-20T22:15:26.8796999Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T22:15:26.8836567Z ##[group]Run rm artifacts.zip 2024-08-20T22:15:26.8836958Z rm artifacts.zip 2024-08-20T22:15:26.8845747Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:26.8846259Z env: 2024-08-20T22:15:26.8846535Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:26.8846993Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:26.8847476Z ##[endgroup] 2024-08-20T22:15:27.0193809Z ##[group]Run df -H 2024-08-20T22:15:27.0194133Z df -H 2024-08-20T22:15:27.0202217Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:27.0202716Z env: 2024-08-20T22:15:27.0202987Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:27.0203433Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:27.0203907Z ##[endgroup] 2024-08-20T22:15:27.0254072Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T22:15:27.0254793Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T22:15:27.0255283Z tmpfs 34G 0 34G 0% /dev/shm 2024-08-20T22:15:27.0255766Z tmpfs 14G 553k 14G 1% /run 2024-08-20T22:15:27.0256231Z /dev/nvme0n1p1 161G 53G 109G 33% / 2024-08-20T22:15:27.0256940Z tmpfs 34G 4.1k 34G 1% /tmp 2024-08-20T22:15:27.0257836Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T22:15:27.0258415Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-08-20T22:15:27.0302843Z Prepare all required actions 2024-08-20T22:15:27.0303298Z Getting action download info 2024-08-20T22:15:27.1408959Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T22:15:27.1409413Z with: 2024-08-20T22:15:27.1409670Z env: 2024-08-20T22:15:27.1409946Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:27.1410390Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:27.1410872Z ##[endgroup] 2024-08-20T22:15:27.1556296Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:15:27.1556746Z with: 2024-08-20T22:15:27.1557015Z name: td_results 2024-08-20T22:15:27.1557336Z s3-bucket: gha-artifacts 2024-08-20T22:15:27.1557689Z region: us-east-1 2024-08-20T22:15:27.1557993Z env: 2024-08-20T22:15:27.1558264Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:27.1558908Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:27.1559409Z ##[endgroup] 2024-08-20T22:15:27.6526118Z (node:52507) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:15:27.6526809Z 2024-08-20T22:15:27.6527089Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:15:27.6527817Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:15:27.6528931Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:15:27.7223560Z Found 0 objects with prefix pytorch/pytorch/10479310721/td_results/ 2024-08-20T22:15:27.7230501Z Artifact download has finished successfully 2024-08-20T22:15:27.7597689Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T22:15:27.7598185Z mkdir -p .additional_ci_files 2024-08-20T22:15:27.7598756Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T22:15:27.7608531Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:27.7609043Z env: 2024-08-20T22:15:27.7609332Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:27.7609807Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:27.7610298Z ##[endgroup] 2024-08-20T22:15:27.7669176Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T22:15:27.7683979Z ##[error]Process completed with exit code 1. 2024-08-20T22:15:27.8019867Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T22:15:27.8020353Z .github/scripts/parse_ref.py 2024-08-20T22:15:27.8028659Z shell: /usr/bin/bash -e {0} 2024-08-20T22:15:27.8029027Z env: 2024-08-20T22:15:27.8029316Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:27.8029779Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:27.8030263Z ##[endgroup] 2024-08-20T22:15:27.8324173Z Prepare all required actions 2024-08-20T22:15:27.8420781Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T22:15:27.8421244Z with: 2024-08-20T22:15:27.8421830Z github-token: *** 2024-08-20T22:15:27.8422146Z env: 2024-08-20T22:15:27.8422428Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:27.8422887Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:27.8423379Z ##[endgroup] 2024-08-20T22:15:27.8540523Z ##[group]Run set -eux 2024-08-20T22:15:27.8540860Z set -eux 2024-08-20T22:15:27.8541453Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T22:15:27.8550393Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:27.8550885Z env: 2024-08-20T22:15:27.8551152Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:27.8551607Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:27.8552248Z GITHUB_TOKEN: *** 2024-08-20T22:15:27.8552560Z ##[endgroup] 2024-08-20T22:15:27.8580592Z + python3 .github/scripts/get_workflow_job_id.py 10479310721 i-02d1d9ee19baacafb 2024-08-20T22:15:28.4828887Z setting job-id=29026299476 2024-08-20T22:15:28.4829923Z setting job-name=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:28.5056247Z Prepare all required actions 2024-08-20T22:15:28.5056691Z Getting action download info 2024-08-20T22:15:28.6127187Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T22:15:28.6127651Z with: 2024-08-20T22:15:28.6128110Z github-token: *** 2024-08-20T22:15:28.6134742Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:15:28.6142605Z job-name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:28.6143467Z env: 2024-08-20T22:15:28.6143754Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:28.6144205Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:28.6144685Z ##[endgroup] 2024-08-20T22:15:28.6244347Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:15:28.6244908Z with: 2024-08-20T22:15:28.6245201Z shell: bash 2024-08-20T22:15:28.6245547Z timeout_minutes: 10 2024-08-20T22:15:28.6245876Z max_attempts: 5 2024-08-20T22:15:28.6246199Z retry_wait_seconds: 30 2024-08-20T22:15:28.6247319Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:15:28.6248494Z polling_interval_seconds: 1 2024-08-20T22:15:28.6248879Z warning_on_retry: true 2024-08-20T22:15:28.6249222Z continue_on_error: false 2024-08-20T22:15:28.6249560Z env: 2024-08-20T22:15:28.6249835Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:28.6250271Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:28.6250913Z GITHUB_TOKEN: *** 2024-08-20T22:15:28.6251228Z ##[endgroup] 2024-08-20T22:15:28.7018131Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:15:28.9410054Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:15:29.1199548Z Collecting requests==2.27.1 2024-08-20T22:15:29.1555566Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-20T22:15:29.4009098Z Collecting pyyaml==6.0.1 2024-08-20T22:15:29.4073433Z Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738 kB) 2024-08-20T22:15:29.4659777Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-20T22:15:29.5373058Z Collecting certifi>=2017.4.17 2024-08-20T22:15:29.5411643Z Downloading certifi-2024.7.4-py3-none-any.whl (162 kB) 2024-08-20T22:15:29.5707312Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-20T22:15:29.8917458Z Collecting charset-normalizer~=2.0.0 2024-08-20T22:15:29.8961739Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-20T22:15:29.9936194Z Installing collected packages: charset-normalizer, certifi, requests, pyyaml 2024-08-20T22:15:30.2881968Z Successfully installed certifi-2024.7.4 charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 2024-08-20T22:15:30.6793308Z Command completed after 1 attempt(s). 2024-08-20T22:15:30.6891225Z ##[group]Run set -x 2024-08-20T22:15:30.6891562Z set -x 2024-08-20T22:15:30.6891856Z  2024-08-20T22:15:30.6892402Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:15:30.6893537Z # in runner workspace 2024-08-20T22:15:30.6894144Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T22:15:30.6903370Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:30.6903874Z env: 2024-08-20T22:15:30.6904149Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:30.6904602Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:30.6905084Z ##[endgroup] 2024-08-20T22:15:30.6934964Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T22:15:30.7183242Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:15:30.7183794Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:15:30.7184271Z echo "Job name: ${JOB_NAME}" 2024-08-20T22:15:30.7184679Z  2024-08-20T22:15:30.7185220Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:15:30.7185940Z # in runner workspace 2024-08-20T22:15:30.7186537Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T22:15:30.7187170Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T22:15:30.7187618Z  --job-name "${JOB_NAME}" \ 2024-08-20T22:15:30.7194711Z  --test-matrix "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-08-20T22:15:30.7201751Z  --selected-test-configs "" \ 2024-08-20T22:15:30.7202211Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T22:15:30.7202628Z  --tag "${TAG}" \ 2024-08-20T22:15:30.7203222Z  --event-name "${EVENT_NAME}" \ 2024-08-20T22:15:30.7203662Z  --schedule "${SCHEDULE}" \ 2024-08-20T22:15:30.7204088Z  --branch "${HEAD_BRANCH}" 2024-08-20T22:15:30.7213062Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:30.7213575Z env: 2024-08-20T22:15:30.7213856Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:30.7214321Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:30.7215012Z GITHUB_TOKEN: *** 2024-08-20T22:15:30.7215807Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:30.7216664Z PR_NUMBER: 2024-08-20T22:15:30.7216981Z TAG: ciflow/inductor/133712 2024-08-20T22:15:30.7217353Z EVENT_NAME: push 2024-08-20T22:15:30.7217665Z SCHEDULE: 2024-08-20T22:15:30.7218124Z HEAD_BRANCH: 2024-08-20T22:15:30.7218423Z ##[endgroup] 2024-08-20T22:15:30.7248792Z Workflow: inductor-periodic 2024-08-20T22:15:30.7249800Z Job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:30.9608404Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T22:15:31.1103937Z ##[group]Run echo "Filtered matrix:" 2024-08-20T22:15:31.1104394Z echo "Filtered matrix:" 2024-08-20T22:15:31.1111057Z echo "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-08-20T22:15:31.1117994Z  2024-08-20T22:15:31.1118278Z echo 2024-08-20T22:15:31.1118661Z echo "Is the current job unstable? False" 2024-08-20T22:15:31.1119117Z  2024-08-20T22:15:31.1119397Z echo 2024-08-20T22:15:31.1119761Z echo "Is keep-going label set? False" 2024-08-20T22:15:31.1120200Z  2024-08-20T22:15:31.1120478Z echo 2024-08-20T22:15:31.1120798Z echo "Renabled issues? " 2024-08-20T22:15:31.1129563Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:31.1130064Z env: 2024-08-20T22:15:31.1130346Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:31.1130802Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:31.1131289Z ##[endgroup] 2024-08-20T22:15:31.1160032Z Filtered matrix: 2024-08-20T22:15:31.1168045Z {include: [{config: dynamo_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-08-20T22:15:31.1175028Z 2024-08-20T22:15:31.1175186Z Is the current job unstable? False 2024-08-20T22:15:31.1175474Z 2024-08-20T22:15:31.1175830Z Is keep-going label set? False 2024-08-20T22:15:31.1176090Z 2024-08-20T22:15:31.1176225Z Renabled issues? 2024-08-20T22:15:31.1338419Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:31.1339476Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:31.1348085Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:31.1348656Z env: 2024-08-20T22:15:31.1349061Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:31.1349682Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:31.1350293Z JOB_TIMEOUT: 240 2024-08-20T22:15:31.1350679Z ##[endgroup] 2024-08-20T22:15:31.1497577Z ##[group]Run set -x 2024-08-20T22:15:31.1497984Z set -x 2024-08-20T22:15:31.1498294Z  2024-08-20T22:15:31.1498657Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T22:15:31.1499329Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T22:15:31.1499897Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T22:15:31.1500420Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T22:15:31.1500847Z else 2024-08-20T22:15:31.1501206Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:15:31.1501632Z fi 2024-08-20T22:15:31.1501923Z  2024-08-20T22:15:31.1502401Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T22:15:31.1503167Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T22:15:31.1503861Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T22:15:31.1504451Z # shellcheck disable=SC2086,SC2090 2024-08-20T22:15:31.1504925Z container_name=$(docker run \ 2024-08-20T22:15:31.1505350Z  ${GPU_FLAG:-} \ 2024-08-20T22:15:31.1505734Z  -e BUILD_ENVIRONMENT \ 2024-08-20T22:15:31.1506143Z  -e PR_NUMBER \ 2024-08-20T22:15:31.1506502Z  -e GITHUB_ACTIONS \ 2024-08-20T22:15:31.1506899Z  -e GITHUB_REPOSITORY \ 2024-08-20T22:15:31.1507307Z  -e GITHUB_WORKFLOW \ 2024-08-20T22:15:31.1507694Z  -e GITHUB_JOB \ 2024-08-20T22:15:31.1508053Z  -e GITHUB_RUN_ID \ 2024-08-20T22:15:31.1508439Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T22:15:31.1508846Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T22:15:31.1509243Z  -e JOB_ID \ 2024-08-20T22:15:31.1509579Z  -e JOB_NAME \ 2024-08-20T22:15:31.1509938Z  -e BASE_SHA \ 2024-08-20T22:15:31.1510284Z  -e BRANCH \ 2024-08-20T22:15:31.1510621Z  -e SHA1 \ 2024-08-20T22:15:31.1510962Z  -e AWS_DEFAULT_REGION \ 2024-08-20T22:15:31.1511368Z  -e IN_WHEEL_TEST \ 2024-08-20T22:15:31.1511750Z  -e SHARD_NUMBER \ 2024-08-20T22:15:31.1512116Z  -e TEST_CONFIG \ 2024-08-20T22:15:31.1512494Z  -e NUM_TEST_SHARDS \ 2024-08-20T22:15:31.1512891Z  -e REENABLED_ISSUES \ 2024-08-20T22:15:31.1513307Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T22:15:31.1513727Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T22:15:31.1514132Z  -e TEST_SHOWLOCALS \ 2024-08-20T22:15:31.1514522Z  -e NO_TEST_TIMEOUT \ 2024-08-20T22:15:31.1514899Z  -e NO_TD \ 2024-08-20T22:15:31.1515237Z  -e TD_DISTRIBUTED \ 2024-08-20T22:15:31.1515620Z  -e PR_LABELS \ 2024-08-20T22:15:31.1516025Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T22:15:31.1516675Z  -e SCCACHE_BUCKET \ 2024-08-20T22:15:31.1517069Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T22:15:31.1517483Z  -e XLA_CUDA \ 2024-08-20T22:15:31.1517895Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T22:15:31.1518408Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T22:15:31.1518926Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T22:15:31.1519444Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T22:15:31.1519913Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T22:15:31.1520466Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T22:15:31.1520900Z  -e DASHBOARD_TAG \ 2024-08-20T22:15:31.1521382Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T22:15:31.1521942Z  --security-opt seccomp=unconfined \ 2024-08-20T22:15:31.1522407Z  --cap-add=SYS_PTRACE \ 2024-08-20T22:15:31.1522797Z  --ipc=host \ 2024-08-20T22:15:31.1523156Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T22:15:31.1523564Z  --tty \ 2024-08-20T22:15:31.1523876Z  --detach \ 2024-08-20T22:15:31.1524363Z  --name="${container_name}" \ 2024-08-20T22:15:31.1524788Z  --user jenkins \ 2024-08-20T22:15:31.1525280Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T22:15:31.1525838Z  -w /var/lib/jenkins/workspace \ 2024-08-20T22:15:31.1526280Z  "${DOCKER_IMAGE}" 2024-08-20T22:15:31.1526639Z ) 2024-08-20T22:15:31.1527050Z # Propagate download.pytorch.org IP to container 2024-08-20T22:15:31.1527964Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T22:15:31.1528929Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T22:15:31.1529846Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T22:15:31.1539441Z shell: /usr/bin/bash -e {0} 2024-08-20T22:15:31.1539803Z env: 2024-08-20T22:15:31.1540077Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:31.1540545Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:31.1541457Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:31.1541969Z PR_NUMBER: 2024-08-20T22:15:31.1542290Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T22:15:31.1542726Z GITHUB_WORKFLOW: inductor-periodic 2024-08-20T22:15:31.1543131Z GITHUB_JOB: test 2024-08-20T22:15:31.1543445Z GITHUB_RUN_ID: 10479310721 2024-08-20T22:15:31.1543808Z GITHUB_RUN_NUMBER: 54565 2024-08-20T22:15:31.1544166Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T22:15:31.1544501Z JOB_ID: 29026299476 2024-08-20T22:15:31.1545291Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:31.1546154Z BRANCH: 2024-08-20T22:15:31.1546546Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:31.1547076Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:31.1547546Z TEST_CONFIG: aot_eager_timm 2024-08-20T22:15:31.1547920Z SHARD_NUMBER: 1 2024-08-20T22:15:31.1548231Z NUM_TEST_SHARDS: 2 2024-08-20T22:15:31.1548555Z REENABLED_ISSUES: 2024-08-20T22:15:31.1548888Z CONTINUE_THROUGH_ERROR: False 2024-08-20T22:15:31.1549276Z VERBOSE_TEST_LOGS: False 2024-08-20T22:15:31.1549636Z TEST_SHOWLOCALS: False 2024-08-20T22:15:31.1549988Z NO_TEST_TIMEOUT: False 2024-08-20T22:15:31.1550321Z NO_TD: False 2024-08-20T22:15:31.1550627Z TD_DISTRIBUTED: False 2024-08-20T22:15:31.1551047Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T22:15:31.1551556Z SCCACHE_S3_KEY_PREFIX: inductor-periodic 2024-08-20T22:15:31.1551982Z SHM_SIZE: 2g 2024-08-20T22:15:31.1553065Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:15:31.1554345Z XLA_CUDA: 2024-08-20T22:15:31.1554823Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:15:31.1555429Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T22:15:31.1555866Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T22:15:31.1556278Z DASHBOARD_TAG: 2024-08-20T22:15:31.1556790Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T22:15:31.1557169Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T22:15:31.1557543Z ##[endgroup] 2024-08-20T22:15:31.1585103Z + [[ aot_eager_timm == \m\u\l\t\i\g\p\u ]] 2024-08-20T22:15:31.1585913Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-08-20T22:15:31.1586439Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:15:31.1594529Z +++ nproc --ignore=2 2024-08-20T22:15:31.1811161Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310721 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:15:37.6289736Z + container_name=dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:15:37.6292641Z + grep download.pytorch.org /etc/hosts 2024-08-20T22:15:37.6295080Z + docker exec -i dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T22:15:37.7565851Z + echo DOCKER_CONTAINER_ID=dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:15:37.7570853Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:15:37.7573826Z + docker exec -t dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T22:15:38.1602613Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T22:15:38.4867834Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T22:15:38.4870160Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T22:15:38.4872560Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T22:15:38.4875007Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T22:15:38.4878512Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T22:15:38.4883047Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.13.1) 2024-08-20T22:15:38.4899550Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T22:15:38.4912675Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T22:15:38.4929547Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.22.4) 2024-08-20T22:15:38.5314478Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T22:15:39.1079481Z Installing collected packages: torch 2024-08-20T22:15:49.2045771Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:15:49.2048272Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T22:15:49.2049391Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T22:15:49.2880474Z + export TERM=vt100 2024-08-20T22:15:49.2881020Z + TERM=vt100 2024-08-20T22:15:49.2882365Z ++ dirname .ci/pytorch/test.sh 2024-08-20T22:15:49.2891689Z + source .ci/pytorch/common.sh 2024-08-20T22:15:49.2895769Z +++ dirname .ci/pytorch/common.sh 2024-08-20T22:15:49.2905994Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T22:15:49.2908254Z +++ declare -f -t trap_add 2024-08-20T22:15:49.2913201Z ++ set -ex 2024-08-20T22:15:49.2913762Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:15:49.2914295Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T22:15:49.2915028Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-08-20T22:15:49.2917845Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T22:15:49.2941450Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T22:15:49.2941911Z + trap_add cleanup_workspace EXIT 2024-08-20T22:15:49.2942329Z + trap_add_cmd=cleanup_workspace 2024-08-20T22:15:49.2942729Z + shift 2024-08-20T22:15:49.2943019Z + for trap_add_name in "$@" 2024-08-20T22:15:49.2950685Z +++ trap -p EXIT 2024-08-20T22:15:49.2954346Z ++ eval 'extract_trap_cmd ' 2024-08-20T22:15:49.2954728Z +++ extract_trap_cmd 2024-08-20T22:15:49.2955096Z +++ printf '%s\n' '' 2024-08-20T22:15:49.2955493Z ++ printf '%s\n' cleanup_workspace 2024-08-20T22:15:49.2958020Z + trap -- ' 2024-08-20T22:15:49.2958370Z cleanup_workspace' EXIT 2024-08-20T22:15:49.2958849Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T22:15:49.9868702Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T22:15:49.9890952Z + echo 'Environment variables:' 2024-08-20T22:15:49.9891359Z Environment variables: 2024-08-20T22:15:49.9891704Z + env 2024-08-20T22:15:49.9903596Z INSTALLED_DB=yes 2024-08-20T22:15:49.9904878Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:15:49.9905495Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:15:49.9905990Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:15:49.9906669Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:15:49.9907257Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:15:49.9907764Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:15:49.9908293Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:15:49.9908881Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:49.9909375Z HOSTNAME=dd0d6f041c6d 2024-08-20T22:15:49.9910430Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:49.9911475Z GITHUB_ACTION=__self 2024-08-20T22:15:49.9911937Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:15:49.9916608Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:15:49.9920897Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:15:49.9921464Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:15:49.9921824Z GITHUB_RUN_NUMBER=54565 2024-08-20T22:15:49.9922172Z TEST_CONFIG=aot_eager_timm 2024-08-20T22:15:49.9922542Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:15:49.9923028Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:15:49.9923508Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:15:49.9923957Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:15:49.9924522Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:15:49.9924930Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:15:49.9925386Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:15:49.9925960Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:15:49.9926434Z GITHUB_REF_TYPE=tag 2024-08-20T22:15:49.9926759Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:15:49.9927147Z NCCL_VERSION=2.17.1-1 2024-08-20T22:15:49.9927545Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:49.9927997Z XLA_CUDA= 2024-08-20T22:15:49.9928493Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:15:49.9930802Z *** 2024-08-20T22:15:49.9931107Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:15:49.9931515Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:15:49.9931881Z GITHUB_ACTIONS=true 2024-08-20T22:15:49.9932221Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:49.9932757Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:15:49.9933303Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:15:49.9933785Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:49.9934304Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:15:49.9934792Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:49.9935728Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:15:49.9936481Z UCC_HOME=/usr 2024-08-20T22:15:49.9936826Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:15:49.9937230Z VERBOSE_TEST_LOGS=False 2024-08-20T22:15:49.9937574Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:15:49.9938045Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:15:49.9938547Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:15:49.9939201Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:15:49.9939581Z SHARD_NUMBER=1 2024-08-20T22:15:49.9939894Z GITHUB_REF_PROTECTED=false 2024-08-20T22:15:49.9940259Z HOME=/var/lib/jenkins 2024-08-20T22:15:49.9940630Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:15:49.9941069Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:15:49.9941557Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:15:49.9942120Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-20T22:15:49.9942534Z CUDA_VERSION=12.1.1 2024-08-20T22:15:49.9942960Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:15:49.9943409Z NUM_TEST_SHARDS=2 2024-08-20T22:15:49.9943717Z UCX_HOME=/usr 2024-08-20T22:15:49.9944237Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:15:49.9945321Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:49.9946706Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:49.9948067Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:49.9949301Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:15:49.9949958Z GITHUB_EVENT_NAME=push 2024-08-20T22:15:49.9950292Z DASHBOARD_TAG= 2024-08-20T22:15:49.9950602Z GITHUB_RUN_ID=10479310721 2024-08-20T22:15:49.9951115Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:15:49.9951665Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:15:49.9952666Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:49.9953581Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:15:49.9953989Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:15:49.9955840Z PR_NUMBER= 2024-08-20T22:15:49.9956134Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:15:49.9956478Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:15:49.9956918Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:15:49.9957379Z TERM=vt100 2024-08-20T22:15:49.9957731Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:15:49.9958153Z INSTALLED_VISION=yes 2024-08-20T22:15:49.9958469Z BRANCH= 2024-08-20T22:15:49.9958861Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:15:49.9959275Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:15:49.9959699Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:15:49.9960472Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:15:49.9961234Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:15:49.9961728Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:15:49.9962201Z REENABLED_ISSUES= 2024-08-20T22:15:49.9962511Z SHLVL=1 2024-08-20T22:15:49.9962781Z MAX_JOBS=14 2024-08-20T22:15:49.9963107Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:15:49.9963462Z NVARCH=x86_64 2024-08-20T22:15:49.9963768Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:15:49.9964237Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:49.9964786Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:15:49.9965290Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:15:49.9965968Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:15:49.9966548Z GITHUB_JOB=test 2024-08-20T22:15:49.9966950Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:15:49.9967513Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:15:49.9968032Z NO_TEST_TIMEOUT=False 2024-08-20T22:15:49.9968367Z TD_DISTRIBUTED=False 2024-08-20T22:15:49.9968746Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:15:49.9969189Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:15:49.9969615Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:15:49.9969979Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:15:49.9970338Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:15:49.9970699Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:15:49.9971741Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:49.9972805Z GITHUB_BASE_REF= 2024-08-20T22:15:49.9973132Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:15:49.9973503Z CI=true 2024-08-20T22:15:49.9973820Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:15:49.9974226Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:15:49.9975975Z JOB_ID=29026299476 2024-08-20T22:15:49.9976290Z INSTALLED_PROTOBUF=yes 2024-08-20T22:15:49.9976619Z GITHUB_HEAD_REF= 2024-08-20T22:15:49.9976914Z GITHUB_ACTION_REF= 2024-08-20T22:15:49.9977350Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:15:49.9977816Z TEST_SHOWLOCALS=False 2024-08-20T22:15:49.9978206Z GITHUB_WORKFLOW=inductor-periodic 2024-08-20T22:15:49.9978610Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:15:49.9979692Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:49.9980516Z NO_TD=False 2024-08-20T22:15:49.9980824Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:15:49.9981343Z _=/usr/bin/env 2024-08-20T22:15:49.9981835Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T22:15:50.0141897Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-20T22:15:50.0143096Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-20T22:15:50.0144212Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-20T22:15:50.0145115Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-20T22:15:50.0145733Z + BUILD_DIR=build 2024-08-20T22:15:50.0146066Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T22:15:50.0146467Z + BUILD_BIN_DIR=build/bin 2024-08-20T22:15:50.0146821Z + SHARD_NUMBER=1 2024-08-20T22:15:50.0147135Z + NUM_TEST_SHARDS=2 2024-08-20T22:15:50.0147462Z + export VALGRIND=ON 2024-08-20T22:15:50.0147794Z + VALGRIND=ON 2024-08-20T22:15:50.0148278Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-08-20T22:15:50.0148837Z + [[ 0 == \1 ]] 2024-08-20T22:15:50.0149226Z + [[ False == \1 ]] 2024-08-20T22:15:50.0149744Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-08-20T22:15:50.0150517Z ++ realpath build/custom_test_artifacts 2024-08-20T22:15:50.0160792Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T22:15:50.0161580Z + [[ -n '' ]] 2024-08-20T22:15:50.0161955Z + echo 'Environment variables' 2024-08-20T22:15:50.0162334Z Environment variables 2024-08-20T22:15:50.0162653Z + env 2024-08-20T22:15:50.0169434Z INSTALLED_DB=yes 2024-08-20T22:15:50.0169983Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:15:50.0170489Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:15:50.0171059Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:15:50.0171965Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:15:50.0172765Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:15:50.0173465Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:15:50.0174175Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:15:50.0174882Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:50.0175427Z HOSTNAME=dd0d6f041c6d 2024-08-20T22:15:50.0176346Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:50.0177153Z GITHUB_ACTION=__self 2024-08-20T22:15:50.0177499Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:15:50.0182129Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:15:50.0186131Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:15:50.0186644Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:15:50.0186993Z GITHUB_RUN_NUMBER=54565 2024-08-20T22:15:50.0187329Z TEST_CONFIG=aot_eager_timm 2024-08-20T22:15:50.0187773Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:15:50.0188253Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:15:50.0188721Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:15:50.0189170Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:15:50.0189570Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:15:50.0189963Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:15:50.0190411Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:15:50.0190909Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:15:50.0191630Z GITHUB_REF_TYPE=tag 2024-08-20T22:15:50.0191952Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:15:50.0192331Z NCCL_VERSION=2.17.1-1 2024-08-20T22:15:50.0192725Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:50.0193439Z XLA_CUDA= 2024-08-20T22:15:50.0193927Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:15:50.0194322Z *** 2024-08-20T22:15:50.0194604Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:15:50.0194999Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:15:50.0195366Z GITHUB_ACTIONS=true 2024-08-20T22:15:50.0195703Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:50.0196207Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:15:50.0196754Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:15:50.0197229Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:50.0197746Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:15:50.0198225Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:50.0199163Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:15:50.0199936Z UCC_HOME=/usr 2024-08-20T22:15:50.0200279Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:15:50.0200842Z VERBOSE_TEST_LOGS=False 2024-08-20T22:15:50.0201236Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:15:50.0201704Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:15:50.0202204Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:15:50.0202667Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:15:50.0203046Z SHARD_NUMBER=1 2024-08-20T22:15:50.0203355Z GITHUB_REF_PROTECTED=false 2024-08-20T22:15:50.0203700Z HOME=/var/lib/jenkins 2024-08-20T22:15:50.0204065Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:15:50.0204501Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:15:50.0204969Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:15:50.0205509Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-20T22:15:50.0205923Z CUDA_VERSION=12.1.1 2024-08-20T22:15:50.0206349Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:15:50.0206799Z NUM_TEST_SHARDS=2 2024-08-20T22:15:50.0207092Z UCX_HOME=/usr 2024-08-20T22:15:50.0207614Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:15:50.0208701Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:50.0210086Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:50.0211460Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:50.0212570Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:15:50.0213219Z GITHUB_EVENT_NAME=push 2024-08-20T22:15:50.0213546Z DASHBOARD_TAG= 2024-08-20T22:15:50.0213843Z GITHUB_RUN_ID=10479310721 2024-08-20T22:15:50.0214308Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:15:50.0214858Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:15:50.0215869Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:50.0216781Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:15:50.0217188Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:15:50.0217568Z PR_NUMBER= 2024-08-20T22:15:50.0217852Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:15:50.0218159Z VALGRIND=ON 2024-08-20T22:15:50.0218457Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:15:50.0219045Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:15:50.0219494Z TERM=vt100 2024-08-20T22:15:50.0219839Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:15:50.0220257Z INSTALLED_VISION=yes 2024-08-20T22:15:50.0220566Z BRANCH= 2024-08-20T22:15:50.0220861Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:15:50.0221297Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:15:50.0221723Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:15:50.0222489Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:15:50.0223386Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:15:50.0223877Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:15:50.0224348Z REENABLED_ISSUES= 2024-08-20T22:15:50.0224639Z SHLVL=1 2024-08-20T22:15:50.0224899Z MAX_JOBS=14 2024-08-20T22:15:50.0225217Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:15:50.0225566Z NVARCH=x86_64 2024-08-20T22:15:50.0225855Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:15:50.0226311Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:50.0226849Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:15:50.0227341Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:15:50.0228003Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:15:50.0228577Z GITHUB_JOB=test 2024-08-20T22:15:50.0228979Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:15:50.0229528Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:15:50.0230049Z NO_TEST_TIMEOUT=False 2024-08-20T22:15:50.0230374Z TD_DISTRIBUTED=False 2024-08-20T22:15:50.0230862Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:15:50.0231292Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:15:50.0231716Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:15:50.0232078Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:15:50.0232424Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:15:50.0232767Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:15:50.0233807Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:50.0234879Z GITHUB_BASE_REF= 2024-08-20T22:15:50.0235205Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:15:50.0235564Z CI=true 2024-08-20T22:15:50.0235880Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:15:50.0236293Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:15:50.0236658Z JOB_ID=29026299476 2024-08-20T22:15:50.0236979Z INSTALLED_PROTOBUF=yes 2024-08-20T22:15:50.0237305Z GITHUB_HEAD_REF= 2024-08-20T22:15:50.0237605Z GITHUB_ACTION_REF= 2024-08-20T22:15:50.0238039Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:15:50.0238505Z TEST_SHOWLOCALS=False 2024-08-20T22:15:50.0238898Z GITHUB_WORKFLOW=inductor-periodic 2024-08-20T22:15:50.0239307Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:15:50.0240219Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_2b7fb297-644e-49b3-936e-5c153fae7d07 2024-08-20T22:15:50.0241060Z NO_TD=False 2024-08-20T22:15:50.0241374Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:15:50.0241729Z _=/usr/bin/env 2024-08-20T22:15:50.0242073Z + echo 'Testing pytorch' 2024-08-20T22:15:50.0242415Z Testing pytorch 2024-08-20T22:15:50.0242736Z + export LANG=C.UTF-8 2024-08-20T22:15:50.0243066Z + LANG=C.UTF-8 2024-08-20T22:15:50.0243356Z + PR_NUMBER= 2024-08-20T22:15:50.0243679Z + [[ aot_eager_timm == \d\e\f\a\u\l\t ]] 2024-08-20T22:15:50.0244148Z + [[ aot_eager_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T22:15:50.0244605Z + [[ aot_eager_timm == \s\l\o\w ]] 2024-08-20T22:15:50.0245217Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-08-20T22:15:50.0245917Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-08-20T22:15:50.0246463Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:15:50.0246940Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:15:50.0247372Z + [[ aot_eager_timm == *crossref* ]] 2024-08-20T22:15:50.0247929Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:15:50.0248563Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-08-20T22:15:50.0249215Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:15:50.0249781Z + pip_install --user ninja==1.10.2 2024-08-20T22:15:50.0250317Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T22:15:51.2794039Z Collecting ninja==1.10.2 2024-08-20T22:15:51.2990550Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T22:15:51.3659456Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T22:15:51.9792290Z Installing collected packages: ninja 2024-08-20T22:15:51.9869758Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T22:15:51.9871048Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T22:15:52.0381526Z Successfully installed ninja-1.10.2 2024-08-20T22:15:52.1172566Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:52.1174737Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:52.1176356Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:15:52.1176868Z + install_tlparse 2024-08-20T22:15:52.1177622Z + pip_install --user tlparse==0.3.25 2024-08-20T22:15:52.1178202Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T22:15:52.5433495Z Collecting tlparse==0.3.25 2024-08-20T22:15:52.5596792Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T22:15:52.6176525Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T22:15:53.2804548Z Installing collected packages: tlparse 2024-08-20T22:15:53.3140510Z Successfully installed tlparse-0.3.25 2024-08-20T22:15:53.3937297Z ++ python -m site --user-base 2024-08-20T22:15:53.4180299Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:53.4182114Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-08-20T22:15:53.4182794Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-08-20T22:15:53.4183460Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:15:53.4184411Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-08-20T22:15:53.4185563Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-08-20T22:15:53.4186524Z + cd test 2024-08-20T22:15:53.4187066Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T22:15:55.0439757Z + [[ aot_eager_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T22:15:55.0440313Z + [[ aot_eager_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T22:15:55.0444962Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T22:15:55.0445534Z + [[ aot_eager_timm == *pr_time_benchmarks* ]] 2024-08-20T22:15:55.0446065Z + [[ aot_eager_timm == *dynamo_eager* ]] 2024-08-20T22:15:55.0446526Z + [[ aot_eager_timm == *aot_eager* ]] 2024-08-20T22:15:55.0447137Z + DYNAMO_BENCHMARK_FLAGS+=(--backend aot_eager) 2024-08-20T22:15:55.0447616Z + [[ aot_eager_timm == *dynamic* ]] 2024-08-20T22:15:55.0448035Z + [[ aot_eager_timm == *cpu* ]] 2024-08-20T22:15:55.0448506Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T22:15:55.0479065Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:15:55.0479763Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-08-20T22:15:55.0481689Z + cd test 2024-08-20T22:15:55.0482542Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T22:15:56.5267303Z PyTorch built with: 2024-08-20T22:15:56.5267898Z - GCC 9.4 2024-08-20T22:15:56.5268371Z - C++ Version: 201703 2024-08-20T22:15:56.5269314Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:15:56.5270410Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:15:56.5271361Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:15:56.5271899Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T22:15:56.5272398Z - NNPACK is enabled 2024-08-20T22:15:56.5272787Z - CPU capability usage: AVX2 2024-08-20T22:15:56.5273203Z - CUDA Runtime 12.1 2024-08-20T22:15:56.5273732Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-08-20T22:15:56.5274342Z - CuDNN 90.1 (built against CUDA 12.4) 2024-08-20T22:15:56.5274786Z - Magma 2.6.1 2024-08-20T22:15:56.5281808Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T22:15:56.5288227Z 2024-08-20T22:15:56.8576637Z + cd test 2024-08-20T22:15:56.8577425Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T22:15:58.2115937Z ATen/Parallel: 2024-08-20T22:15:58.2116322Z at::get_num_threads() : 8 2024-08-20T22:15:58.2116836Z at::get_num_interop_threads() : 16 2024-08-20T22:15:58.2117279Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:15:58.2117699Z omp_get_max_threads() : 8 2024-08-20T22:15:58.2118613Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:15:58.2119415Z mkl_get_max_threads() : 8 2024-08-20T22:15:58.2120048Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:15:58.2120690Z std::thread::hardware_concurrency() : 16 2024-08-20T22:15:58.2121118Z Environment variables: 2024-08-20T22:15:58.2121472Z OMP_NUM_THREADS : [not set] 2024-08-20T22:15:58.2121844Z MKL_NUM_THREADS : [not set] 2024-08-20T22:15:58.2122219Z ATen parallel backend: OpenMP 2024-08-20T22:15:58.2122477Z 2024-08-20T22:15:58.4968392Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:15:58.4969135Z + [[ aot_eager_timm == *backward* ]] 2024-08-20T22:15:58.4969683Z + [[ aot_eager_timm == *xla* ]] 2024-08-20T22:15:58.4970086Z + [[ aot_eager_timm == *executorch* ]] 2024-08-20T22:15:58.4970535Z + [[ aot_eager_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T22:15:58.4971154Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:15:58.4971688Z + [[ aot_eager_timm == distributed ]] 2024-08-20T22:15:58.4972137Z + [[ aot_eager_timm == *inductor_distributed* ]] 2024-08-20T22:15:58.4972668Z + [[ aot_eager_timm == *inductor-halide* ]] 2024-08-20T22:15:58.4973229Z + [[ aot_eager_timm == *inductor-micro-benchmark* ]] 2024-08-20T22:15:58.4973721Z + [[ aot_eager_timm == *huggingface* ]] 2024-08-20T22:15:58.4974141Z + [[ aot_eager_timm == *timm* ]] 2024-08-20T22:15:58.4974523Z + install_torchvision 2024-08-20T22:15:58.4974851Z + local orig_preload 2024-08-20T22:15:58.4975170Z + local commit 2024-08-20T22:15:58.4975478Z ++ get_pinned_commit vision 2024-08-20T22:15:58.4975864Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T22:15:58.4993928Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:15:58.4994478Z + orig_preload= 2024-08-20T22:15:58.4994835Z + '[' -n '' ']' 2024-08-20T22:15:58.4995686Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:15:58.4997130Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:15:58.8360016Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:15:58.8363505Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-zmae17fk 2024-08-20T22:15:58.8394977Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-zmae17fk 2024-08-20T22:16:00.2881993Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T22:16:00.2909245Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:01.6149423Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:01.9159516Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:04.4011904Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-20T22:16:04.4044241Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2024-08-20T22:16:04.4048399Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T22:16:04.4052718Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T22:16:04.4123661Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T22:16:04.4128255Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T22:16:04.4131348Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T22:16:04.4134512Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T22:16:04.4137892Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T22:16:04.4143361Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-20T22:16:04.4157647Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T22:16:04.4635875Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T22:16:04.4699077Z Building wheels for collected packages: torchvision 2024-08-20T22:17:21.2938530Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-08-20T22:17:21.2971945Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2116360 sha256=6927666bb4cf7d7e035f335a76191ed1b094e2994ff30502565e2b3df0c800a4 2024-08-20T22:17:21.2975393Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-20T22:17:21.3008889Z Successfully built torchvision 2024-08-20T22:17:21.7854956Z Installing collected packages: torchvision 2024-08-20T22:17:22.2010409Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T22:17:22.3432046Z + '[' -n '' ']' 2024-08-20T22:17:22.3432414Z + id=0 2024-08-20T22:17:22.3434904Z + test_dynamo_benchmark timm_models 0 2024-08-20T22:17:22.3435510Z ++ pwd 2024-08-20T22:17:22.3439324Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:17:22.3440011Z + local suite=timm_models 2024-08-20T22:17:22.3440354Z + shift 2024-08-20T22:17:22.3440622Z + local shard_id=0 2024-08-20T22:17:22.3440927Z + shift 2024-08-20T22:17:22.3441237Z + [[ aot_eager_timm == *perf_compare* ]] 2024-08-20T22:17:22.3441663Z + [[ aot_eager_timm == *perf* ]] 2024-08-20T22:17:22.3442108Z + [[ aot_eager_timm == *cpu* ]] 2024-08-20T22:17:22.3446325Z + [[ aot_eager_timm == *aot_inductor* ]] 2024-08-20T22:17:22.3447325Z + test_single_dynamo_benchmark inference timm_models 0 --inference --bfloat16 2024-08-20T22:17:22.3448082Z ++ pwd 2024-08-20T22:17:22.3449746Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:17:22.3450437Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:17:22.3474594Z + local name=inference 2024-08-20T22:17:22.3475222Z + shift 2024-08-20T22:17:22.3475659Z + local suite=timm_models 2024-08-20T22:17:22.3476146Z + shift 2024-08-20T22:17:22.3476429Z + local shard_id=0 2024-08-20T22:17:22.3476772Z + shift 2024-08-20T22:17:22.3477134Z + partition_flags=() 2024-08-20T22:17:22.3477577Z + local partition_flags 2024-08-20T22:17:22.3478103Z + [[ -n 2 ]] 2024-08-20T22:17:22.3478462Z + [[ -n 0 ]] 2024-08-20T22:17:22.3479120Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T22:17:22.3479781Z + [[ aot_eager_timm == *perf_compare* ]] 2024-08-20T22:17:22.3480218Z + [[ aot_eager_timm == *perf* ]] 2024-08-20T22:17:22.3480630Z + [[ aot_eager_timm == *aot_inductor* ]] 2024-08-20T22:17:22.3481060Z + [[ aot_eager_timm == *_avx2* ]] 2024-08-20T22:17:22.3481472Z + [[ aot_eager_timm == *_avx512* ]] 2024-08-20T22:17:22.3483115Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --backend aot_eager --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-08-20T22:17:28.8105036Z 2024-08-20T22:17:29.1235220Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:17:29.1235657Z 2024-08-20T22:17:29.2356432Z model.safetensors: 0% 0.00/95.5M [00:00 will be ignored 2024-08-20T22:29:10.4541045Z pass 2024-08-20T22:29:10.5497141Z TIMING: entire_frame_compile:27.11774 backend_compile:14.87251 total_wall_time:27.11774 2024-08-20T22:29:10.5501403Z STATS: call_* op count: 989 | FakeTensorMode.__torch_dispatch__:39822 | FakeTensor.__torch_dispatch__:3842 | ProxyTorchDispatchMode.__torch_dispatch__:18627 2024-08-20T22:29:10.5502577Z Dynamo produced 3 graphs covering 989 ops with 7 graph breaks (5 unique) 2024-08-20T22:29:15.2172403Z 2024-08-20T22:29:16.1701174Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:16.1701697Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:16.1702147Z cuda train botnet26t_256 2024-08-20T22:29:28.0327574Z pass 2024-08-20T22:29:28.0508862Z TIMING: entire_frame_compile:10.34549 backend_compile:4.99468 total_wall_time:10.34549 2024-08-20T22:29:28.0510431Z STATS: call_* op count: 264 | FakeTensorMode.__torch_dispatch__:17305 | FakeTensor.__torch_dispatch__:40 | ProxyTorchDispatchMode.__torch_dispatch__:5527 2024-08-20T22:29:28.0511580Z Dynamo produced 2 graphs covering 264 ops with 6 graph breaks (5 unique) 2024-08-20T22:29:32.3335108Z 2024-08-20T22:29:36.3258130Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:36.3258783Z loading model: 0it [00:03, ?it/s] 2024-08-20T22:29:36.3259586Z cuda train cait_m36_384 2024-08-20T22:29:36.3266324Z Traceback (most recent call last): 2024-08-20T22:29:36.3267438Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2507, in validate_model 2024-08-20T22:29:36.3268574Z self.model_iter_fn(model, example_inputs) 2024-08-20T22:29:36.3269801Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in forward_and_backward_pass 2024-08-20T22:29:36.3270987Z pred = mod(*cloned_inputs) 2024-08-20T22:29:36.3273110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:29:36.3274461Z return self._call_impl(*args, **kwargs) 2024-08-20T22:29:36.3275862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:29:36.3277459Z return forward_call(*args, **kwargs) 2024-08-20T22:29:36.3278738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 438, in forward 2024-08-20T22:29:36.3279833Z x = self.forward_features(x) 2024-08-20T22:29:36.3281107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 423, in forward_features 2024-08-20T22:29:36.3282417Z x = self.blocks(x) 2024-08-20T22:29:36.3283736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:29:36.3285041Z return self._call_impl(*args, **kwargs) 2024-08-20T22:29:36.3286419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:29:36.3287634Z return forward_call(*args, **kwargs) 2024-08-20T22:29:36.3289001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/container.py", line 250, in forward 2024-08-20T22:29:36.3290221Z input = module(input) 2024-08-20T22:29:36.3291772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:29:36.3293375Z return self._call_impl(*args, **kwargs) 2024-08-20T22:29:36.3294817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:29:36.3296070Z return forward_call(*args, **kwargs) 2024-08-20T22:29:36.3297322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 199, in forward 2024-08-20T22:29:36.3298613Z x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x))) 2024-08-20T22:29:36.3300480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:29:36.3301808Z return self._call_impl(*args, **kwargs) 2024-08-20T22:29:36.3303204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:29:36.3304450Z return forward_call(*args, **kwargs) 2024-08-20T22:29:36.3305721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 153, in forward 2024-08-20T22:29:36.3306924Z x = (attn @ v).transpose(1, 2).reshape(B, N, C) 2024-08-20T22:29:36.3311887Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 21.98 GiB of which 16.44 MiB is free. Process 58207 has 21.95 GiB memory in use. Of the allocated memory 21.21 GiB is allocated by PyTorch, and 430.66 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 2024-08-20T22:29:36.3316138Z 2024-08-20T22:29:36.3316582Z The above exception was the direct cause of the following exception: 2024-08-20T22:29:36.3317274Z 2024-08-20T22:29:36.3317493Z Traceback (most recent call last): 2024-08-20T22:29:36.3318457Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T22:29:36.3319462Z ) = runner.load_model( 2024-08-20T22:29:36.3320455Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 328, in load_model 2024-08-20T22:29:36.3321604Z self.validate_model(model, example_inputs) 2024-08-20T22:29:36.3322747Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2509, in validate_model 2024-08-20T22:29:36.3323926Z raise RuntimeError("Eager run failed") from e 2024-08-20T22:29:36.3324630Z RuntimeError: Eager run failed 2024-08-20T22:29:36.3324999Z 2024-08-20T22:29:36.3325175Z eager_fail_to_run 2024-08-20T22:29:40.2651496Z 2024-08-20T22:29:41.1932273Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:41.1932745Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:41.1933190Z cuda train coat_lite_mini 2024-08-20T22:29:53.6780074Z pass 2024-08-20T22:29:53.6975979Z TIMING: entire_frame_compile:11.07902 backend_compile:6.37451 total_wall_time:11.07902 2024-08-20T22:29:53.6977463Z STATS: call_* op count: 456 | FakeTensorMode.__torch_dispatch__:24347 | FakeTensor.__torch_dispatch__:38 | ProxyTorchDispatchMode.__torch_dispatch__:10667 2024-08-20T22:29:53.6978580Z Dynamo produced 2 graphs covering 456 ops with 6 graph breaks (5 unique) 2024-08-20T22:29:58.1531504Z 2024-08-20T22:29:59.7857900Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:59.7858367Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:29:59.7858810Z cuda train convit_base 2024-08-20T22:29:59.8178410Z WARNING:common:fp64 golden ref were not generated for convit_base. Setting accuracy check to cosine 2024-08-20T22:30:24.4433917Z pass 2024-08-20T22:30:24.4929404Z TIMING: entire_frame_compile:23.63022 backend_compile:14.14149 total_wall_time:23.63022 2024-08-20T22:30:24.4931999Z STATS: call_* op count: 1142 | FakeTensorMode.__torch_dispatch__:59357 | FakeTensor.__torch_dispatch__:184 | ProxyTorchDispatchMode.__torch_dispatch__:28022 2024-08-20T22:30:24.4933927Z Dynamo produced 3 graphs covering 1142 ops with 7 graph breaks (5 unique) 2024-08-20T22:30:29.1707854Z 2024-08-20T22:30:30.0155842Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:30.0156573Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:30.0157157Z cuda train convmixer_768_32 2024-08-20T22:30:54.0013508Z pass 2024-08-20T22:30:54.0703758Z TIMING: entire_frame_compile:17.81395 backend_compile:8.35544 total_wall_time:17.81395 2024-08-20T22:30:54.0705165Z STATS: call_* op count: 300 | FakeTensorMode.__torch_dispatch__:29374 | FakeTensor.__torch_dispatch__:26 | ProxyTorchDispatchMode.__torch_dispatch__:9010 2024-08-20T22:30:54.0706284Z Dynamo produced 2 graphs covering 300 ops with 5 graph breaks (4 unique) 2024-08-20T22:30:58.5552662Z 2024-08-20T22:31:00.2778553Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:31:00.2779457Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:31:00.2780138Z cuda train convnext_base 2024-08-20T22:31:22.2800071Z W0820 22:31:22.279000 5180 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:31:53.9905275Z pass 2024-08-20T22:31:54.0770604Z TIMING: entire_frame_compile:48.39125 backend_compile:25.07796 total_wall_time:48.39125 2024-08-20T22:31:54.0771989Z STATS: call_* op count: 1508 | FakeTensorMode.__torch_dispatch__:63187 | FakeTensor.__torch_dispatch__:6317 | ProxyTorchDispatchMode.__torch_dispatch__:29746 2024-08-20T22:31:54.0777153Z Dynamo produced 3 graphs covering 1508 ops with 7 graph breaks (5 unique) 2024-08-20T22:31:59.2274799Z 2024-08-20T22:31:59.9925397Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:31:59.9925925Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:31:59.9926365Z cuda train crossvit_9_240 2024-08-20T22:32:17.0435245Z W0820 22:32:17.042000 5276 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:32:40.8567651Z pass 2024-08-20T22:32:40.8749418Z TIMING: entire_frame_compile:38.79522 backend_compile:21.05016 total_wall_time:38.79522 2024-08-20T22:32:40.8752379Z STATS: call_* op count: 1312 | FakeTensorMode.__torch_dispatch__:59175 | FakeTensor.__torch_dispatch__:5100 | ProxyTorchDispatchMode.__torch_dispatch__:27501 2024-08-20T22:32:40.8753562Z Dynamo produced 3 graphs covering 1312 ops with 7 graph breaks (5 unique) 2024-08-20T22:32:45.7294959Z 2024-08-20T22:32:46.9205239Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:32:46.9205717Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:32:46.9211610Z cuda train cspdarknet53 2024-08-20T22:33:08.2292528Z W0820 22:33:08.228000 5371 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:33:29.3353368Z pass 2024-08-20T22:33:29.3644813Z TIMING: entire_frame_compile:39.11467 backend_compile:18.50221 total_wall_time:39.11467 2024-08-20T22:33:29.3647564Z STATS: call_* op count: 950 | FakeTensorMode.__torch_dispatch__:47017 | FakeTensor.__torch_dispatch__:3808 | ProxyTorchDispatchMode.__torch_dispatch__:17131 2024-08-20T22:33:29.3648725Z Dynamo produced 3 graphs covering 950 ops with 7 graph breaks (5 unique) 2024-08-20T22:33:34.2398469Z 2024-08-20T22:33:35.8870780Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:33:35.8871496Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:33:35.8872184Z cuda train deit_base_distilled_patch16_224 2024-08-20T22:33:48.8477891Z W0820 22:33:48.846000 5467 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:34:02.5109401Z pass 2024-08-20T22:34:02.6008601Z TIMING: entire_frame_compile:21.65497 backend_compile:11.69673 total_wall_time:21.65497 2024-08-20T22:34:02.6012234Z STATS: call_* op count: 753 | FakeTensorMode.__torch_dispatch__:31408 | FakeTensor.__torch_dispatch__:3006 | ProxyTorchDispatchMode.__torch_dispatch__:14631 2024-08-20T22:34:02.6013716Z Dynamo produced 3 graphs covering 753 ops with 7 graph breaks (5 unique) 2024-08-20T22:34:07.1262292Z 2024-08-20T22:34:08.4301381Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:34:08.4301868Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:34:08.4302317Z cuda train dla102 2024-08-20T22:34:38.3327409Z W0820 22:34:38.331000 5563 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:35:11.1022686Z pass 2024-08-20T22:35:11.1332311Z TIMING: entire_frame_compile:58.79261 backend_compile:29.03273 total_wall_time:58.79261 2024-08-20T22:35:11.1333992Z STATS: call_* op count: 1457 | FakeTensorMode.__torch_dispatch__:72709 | FakeTensor.__torch_dispatch__:5872 | ProxyTorchDispatchMode.__torch_dispatch__:26824 2024-08-20T22:35:11.1335724Z Dynamo produced 3 graphs covering 1457 ops with 7 graph breaks (5 unique) 2024-08-20T22:35:16.4966741Z 2024-08-20T22:35:18.2536000Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:35:18.2536722Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:35:18.2537417Z cuda train dm_nfnet_f0 2024-08-20T22:35:37.9122098Z pass 2024-08-20T22:35:37.9634603Z TIMING: entire_frame_compile:16.48254 backend_compile:7.93034 total_wall_time:16.48254 2024-08-20T22:35:37.9635836Z STATS: call_* op count: 594 | FakeTensorMode.__torch_dispatch__:27882 | ProxyTorchDispatchMode.__torch_dispatch__:10519 | FakeTensor.__torch_dispatch__:22 2024-08-20T22:35:37.9636959Z Dynamo produced 2 graphs covering 594 ops with 6 graph breaks (5 unique) 2024-08-20T22:35:42.3676513Z 2024-08-20T22:35:43.9668820Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:35:43.9669299Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:35:43.9669745Z cuda train dpn107 2024-08-20T22:36:22.8197161Z pass 2024-08-20T22:36:22.8981178Z TIMING: entire_frame_compile:32.00477 backend_compile:14.91238 total_wall_time:32.00477 2024-08-20T22:36:22.8982415Z STATS: call_* op count: 634 | FakeTensorMode.__torch_dispatch__:53766 | FakeTensor.__torch_dispatch__:100 | ProxyTorchDispatchMode.__torch_dispatch__:16915 2024-08-20T22:36:22.8983982Z Dynamo produced 2 graphs covering 634 ops with 6 graph breaks (5 unique) 2024-08-20T22:36:27.7457502Z 2024-08-20T22:36:28.6729243Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:36:28.6729905Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:36:28.6730514Z cuda train eca_botnext26ts_256 2024-08-20T22:36:41.0145403Z W0820 22:36:41.013000 5850 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:36:51.9862238Z pass 2024-08-20T22:36:52.0041078Z TIMING: entire_frame_compile:21.70962 backend_compile:11.15312 total_wall_time:21.70962 2024-08-20T22:36:52.0042491Z STATS: call_* op count: 634 | FakeTensorMode.__torch_dispatch__:29043 | FakeTensor.__torch_dispatch__:2110 | ProxyTorchDispatchMode.__torch_dispatch__:11488 2024-08-20T22:36:52.0044057Z Dynamo produced 3 graphs covering 634 ops with 7 graph breaks (5 unique) 2024-08-20T22:36:56.5596981Z 2024-08-20T22:36:57.5114592Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:36:57.5115068Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:36:57.5115511Z cuda train eca_halonext26ts 2024-08-20T22:37:10.2569812Z W0820 22:37:10.256000 5946 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:37:21.7529248Z pass 2024-08-20T22:37:21.7675100Z TIMING: entire_frame_compile:22.60196 backend_compile:11.39358 total_wall_time:22.60196 2024-08-20T22:37:21.7676530Z STATS: call_* op count: 666 | FakeTensorMode.__torch_dispatch__:30023 | FakeTensor.__torch_dispatch__:2160 | ProxyTorchDispatchMode.__torch_dispatch__:11921 2024-08-20T22:37:21.7681011Z Dynamo produced 3 graphs covering 666 ops with 7 graph breaks (5 unique) 2024-08-20T22:37:26.2967004Z 2024-08-20T22:37:27.1079174Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:37:27.1080065Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:37:27.1081647Z cuda train ese_vovnet19b_dw 2024-08-20T22:37:36.3723384Z W0820 22:37:36.371000 6042 torch/_logging/_internal.py:1058] [7/0] Profiler function will be ignored 2024-08-20T22:37:46.9582207Z pass 2024-08-20T22:37:46.9663587Z TIMING: entire_frame_compile:18.5808 backend_compile:9.01306 total_wall_time:18.5808 2024-08-20T22:37:46.9665164Z STATS: call_* op count: 434 | FakeTensorMode.__torch_dispatch__:20624 | FakeTensor.__torch_dispatch__:1842 | ProxyTorchDispatchMode.__torch_dispatch__:7887 2024-08-20T22:37:46.9666436Z Dynamo produced 4 graphs covering 434 ops with 7 graph breaks (5 unique) 2024-08-20T22:37:51.4744825Z 2024-08-20T22:37:52.3576898Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:37:52.3577352Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:37:52.3577820Z cuda train fbnetc_100 2024-08-20T22:38:09.9539290Z W0820 22:38:09.953000 6138 torch/_logging/_internal.py:1058] [7/0] Profiler function will be ignored 2024-08-20T22:38:30.3876622Z pass 2024-08-20T22:38:30.3949275Z TIMING: entire_frame_compile:36.35369 backend_compile:17.77558 total_wall_time:36.35369 2024-08-20T22:38:30.3950499Z STATS: call_* op count: 870 | FakeTensorMode.__torch_dispatch__:44545 | FakeTensor.__torch_dispatch__:3794 | ProxyTorchDispatchMode.__torch_dispatch__:16037 2024-08-20T22:38:30.3951633Z Dynamo produced 4 graphs covering 870 ops with 7 graph breaks (5 unique) 2024-08-20T22:38:35.2051970Z 2024-08-20T22:38:36.2826506Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:38:36.2827103Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:38:36.2827650Z cuda train fbnetv3_b 2024-08-20T22:39:03.7978668Z pass 2024-08-20T22:39:03.8113193Z TIMING: entire_frame_compile:26.06031 backend_compile:11.5827 total_wall_time:26.06031 2024-08-20T22:39:03.8114439Z STATS: call_* op count: 459 | FakeTensorMode.__torch_dispatch__:40513 | FakeTensor.__torch_dispatch__:98 | ProxyTorchDispatchMode.__torch_dispatch__:11990 2024-08-20T22:39:03.8115780Z Dynamo produced 2 graphs covering 459 ops with 6 graph breaks (5 unique) 2024-08-20T22:39:08.5481559Z 2024-08-20T22:39:09.7839777Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:39:09.7840892Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:39:09.7844479Z cuda train gernet_l 2024-08-20T22:39:27.5632959Z pass 2024-08-20T22:39:27.5863283Z TIMING: entire_frame_compile:15.65846 backend_compile:7.1043 total_wall_time:15.65846 2024-08-20T22:39:27.5864670Z STATS: call_* op count: 250 | FakeTensorMode.__torch_dispatch__:23101 | FakeTensor.__torch_dispatch__:38 | ProxyTorchDispatchMode.__torch_dispatch__:6598 2024-08-20T22:39:27.5865792Z Dynamo produced 2 graphs covering 250 ops with 6 graph breaks (5 unique) 2024-08-20T22:39:31.9798315Z 2024-08-20T22:39:32.9145038Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:39:32.9145557Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:39:32.9146036Z cuda train ghostnet_100 2024-08-20T22:39:58.0588253Z pass 2024-08-20T22:39:58.0660864Z TIMING: entire_frame_compile:23.92354 backend_compile:11.17008 total_wall_time:23.92354 2024-08-20T22:39:58.0663638Z STATS: call_* op count: 405 | FakeTensorMode.__torch_dispatch__:38576 | FakeTensor.__torch_dispatch__:74 | ProxyTorchDispatchMode.__torch_dispatch__:11967 2024-08-20T22:39:58.0664744Z Dynamo produced 2 graphs covering 405 ops with 6 graph breaks (5 unique) 2024-08-20T22:40:02.7731571Z 2024-08-20T22:40:04.0578014Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:40:04.0578546Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:40:04.0582168Z cuda train gluon_inception_v3 2024-08-20T22:40:30.9061902Z W0820 22:40:30.905000 6520 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:41:00.4685211Z pass 2024-08-20T22:41:00.4923980Z TIMING: entire_frame_compile:53.35193 backend_compile:25.70308 total_wall_time:53.35193 2024-08-20T22:41:00.4925220Z STATS: call_* op count: 1281 | FakeTensorMode.__torch_dispatch__:65414 | FakeTensor.__torch_dispatch__:5358 | ProxyTorchDispatchMode.__torch_dispatch__:24031 2024-08-20T22:41:00.4928181Z Dynamo produced 3 graphs covering 1281 ops with 7 graph breaks (5 unique) 2024-08-20T22:41:05.8200265Z 2024-08-20T22:41:06.7461931Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:41:06.7462451Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:41:06.7462901Z cuda train gmixer_24_224 2024-08-20T22:41:25.2061779Z pass 2024-08-20T22:41:25.2240404Z TIMING: entire_frame_compile:16.0357 backend_compile:8.87198 total_wall_time:16.0357 2024-08-20T22:41:25.2241916Z STATS: call_* op count: 587 | FakeTensorMode.__torch_dispatch__:35006 | FakeTensor.__torch_dispatch__:218 | ProxyTorchDispatchMode.__torch_dispatch__:16426 2024-08-20T22:41:25.2243088Z Dynamo produced 2 graphs covering 587 ops with 6 graph breaks (5 unique) 2024-08-20T22:41:29.6995672Z 2024-08-20T22:41:30.5543767Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:41:30.5544498Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:41:30.5545140Z cuda train gmlp_s16_224 2024-08-20T22:41:48.0230803Z W0820 22:41:48.022000 6712 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:42:14.5868044Z pass 2024-08-20T22:42:14.6125910Z TIMING: entire_frame_compile:41.23284 backend_compile:22.08412 total_wall_time:41.23284 2024-08-20T22:42:14.6129169Z STATS: call_* op count: 1396 | FakeTensorMode.__torch_dispatch__:60960 | FakeTensor.__torch_dispatch__:5696 | ProxyTorchDispatchMode.__torch_dispatch__:28113 2024-08-20T22:42:14.6130341Z Dynamo produced 3 graphs covering 1396 ops with 7 graph breaks (5 unique) 2024-08-20T22:42:19.5552925Z 2024-08-20T22:42:21.2069837Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:42:21.2070320Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:42:21.2070800Z cuda train hrnet_w18 2024-08-20T22:43:49.3926464Z pass 2024-08-20T22:43:49.4354975Z TIMING: entire_frame_compile:82.69359 backend_compile:36.49737 total_wall_time:82.69359 2024-08-20T22:43:49.4358032Z STATS: call_* op count: 1475 | FakeTensorMode.__torch_dispatch__:135134 | FakeTensor.__torch_dispatch__:46 | ProxyTorchDispatchMode.__torch_dispatch__:39732 2024-08-20T22:43:49.4359202Z Dynamo produced 2 graphs covering 1475 ops with 5 graph breaks (4 unique) 2024-08-20T22:43:55.7147744Z 2024-08-20T22:43:56.9643324Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:43:56.9643904Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:43:56.9647794Z cuda train inception_v3 2024-08-20T22:44:23.9831817Z pass 2024-08-20T22:44:24.0073553Z TIMING: entire_frame_compile:24.41325 backend_compile:11.08661 total_wall_time:24.41325 2024-08-20T22:44:24.0075219Z STATS: call_* op count: 412 | FakeTensorMode.__torch_dispatch__:38654 | FakeTensor.__torch_dispatch__:46 | ProxyTorchDispatchMode.__torch_dispatch__:11234 2024-08-20T22:44:24.0076525Z Dynamo produced 2 graphs covering 412 ops with 6 graph breaks (5 unique) 2024-08-20T22:44:28.6547970Z 2024-08-20T22:44:30.2588451Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:44:30.2589134Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:44:30.2590548Z cuda train jx_nest_base 2024-08-20T22:44:55.7182518Z W0820 22:44:55.717000 7000 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:45:23.0048232Z pass 2024-08-20T22:45:23.0710502Z TIMING: entire_frame_compile:46.46403 backend_compile:25.80377 total_wall_time:46.46403 2024-08-20T22:45:23.0713331Z STATS: call_* op count: 1671 | FakeTensorMode.__torch_dispatch__:74392 | FakeTensor.__torch_dispatch__:5630 | ProxyTorchDispatchMode.__torch_dispatch__:34323 2024-08-20T22:45:23.0714524Z Dynamo produced 3 graphs covering 1671 ops with 7 graph breaks (5 unique) 2024-08-20T22:45:28.1591788Z 2024-08-20T22:45:28.8689527Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:45:28.8690031Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:45:28.8696032Z cuda train lcnet_050 2024-08-20T22:45:37.4506291Z E0820 22:45:37.449000 7096 torch/_dynamo/utils.py:1688] RMSE (res-fp64): 0.00769, (ref-fp64): 0.00017 and shape=torch.Size([32]). res.dtype: torch.float32, multiplier: 3.000000, tol: 0.010000, use_larger_multiplier_for_smaller_tensor: 0 2024-08-20T22:45:37.4508020Z E0820 22:45:37.450000 7096 torch/_dynamo/utils.py:1580] Accuracy failed for key name blocks.1.0.bn2.weight.grad 2024-08-20T22:45:37.4525423Z fail_accuracy 2024-08-20T22:45:37.4548335Z TIMING: entire_frame_compile:8.08859 backend_compile:3.96313 total_wall_time:8.08859 2024-08-20T22:45:37.4549539Z STATS: call_* op count: 129 | FakeTensorMode.__torch_dispatch__:12156 | FakeTensor.__torch_dispatch__:58 | ProxyTorchDispatchMode.__torch_dispatch__:3329 2024-08-20T22:45:37.4550681Z Dynamo produced 2 graphs covering 129 ops with 6 graph breaks (5 unique) 2024-08-20T22:45:41.7747375Z 2024-08-20T22:45:42.8432150Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:45:42.8432643Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:45:42.8433082Z cuda train levit_128 2024-08-20T22:46:03.8221754Z W0820 22:46:03.821000 7192 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:46:22.5349610Z pass 2024-08-20T22:46:22.5408940Z TIMING: entire_frame_compile:38.12568 backend_compile:19.84167 total_wall_time:38.12568 2024-08-20T22:46:22.5410163Z STATS: call_* op count: 1261 | FakeTensorMode.__torch_dispatch__:59537 | FakeTensor.__torch_dispatch__:3902 | ProxyTorchDispatchMode.__torch_dispatch__:23437 2024-08-20T22:46:22.5411433Z Dynamo produced 3 graphs covering 1261 ops with 7 graph breaks (5 unique) 2024-08-20T22:46:27.5088570Z 2024-08-20T22:46:28.7366196Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:46:28.7366745Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:46:28.7367214Z cuda train mixer_b16_224 2024-08-20T22:46:39.4266449Z W0820 22:46:39.425000 7287 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:46:52.4738373Z pass 2024-08-20T22:46:52.5431905Z TIMING: entire_frame_compile:19.90099 backend_compile:10.99154 total_wall_time:19.90099 2024-08-20T22:46:52.5433143Z STATS: call_* op count: 670 | FakeTensorMode.__torch_dispatch__:29576 | FakeTensor.__torch_dispatch__:2868 | ProxyTorchDispatchMode.__torch_dispatch__:13749 2024-08-20T22:46:52.5437638Z Dynamo produced 3 graphs covering 670 ops with 7 graph breaks (5 unique) 2024-08-20T22:46:53.8718488Z accuracy pass_rate=93.33% 2024-08-20T22:46:53.8721823Z calls_captured gmean=0.00x mean=779.033x 2024-08-20T22:46:53.8725482Z unique_graphs gmean=0.00x mean=2.533x 2024-08-20T22:46:53.8729700Z graph_breaks gmean=0.00x mean=6.267x 2024-08-20T22:46:53.8733654Z unique_graph_breaks gmean=0.00x mean=4.767x 2024-08-20T22:46:53.8737826Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T22:46:53.8743906Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T22:46:53.8747524Z cudagraph_skips gmean=0.00x mean=0.000x 2024-08-20T22:46:54.5480655Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/aot_eager_timm_training.csv 2024-08-20T22:46:54.8116433Z adv_inception_v3 PASS 2024-08-20T22:46:54.8120381Z beit_base_patch16_224 PASS 2024-08-20T22:46:54.8125254Z botnet26t_256 PASS 2024-08-20T22:46:54.8130392Z cait_m36_384 XFAIL 2024-08-20T22:46:54.8135282Z coat_lite_mini PASS 2024-08-20T22:46:54.8140496Z convit_base PASS 2024-08-20T22:46:54.8145651Z convmixer_768_32 PASS 2024-08-20T22:46:54.8150351Z convnext_base PASS 2024-08-20T22:46:54.8155889Z crossvit_9_240 PASS 2024-08-20T22:46:54.8160451Z cspdarknet53 PASS 2024-08-20T22:46:54.8165354Z deit_base_distilled_patch16_224 PASS 2024-08-20T22:46:54.8170371Z dla102 PASS 2024-08-20T22:46:54.8175352Z dm_nfnet_f0 PASS 2024-08-20T22:46:54.8180938Z dpn107 PASS 2024-08-20T22:46:54.8185822Z eca_botnext26ts_256 PASS 2024-08-20T22:46:54.8190869Z eca_halonext26ts PASS 2024-08-20T22:46:54.8197553Z ese_vovnet19b_dw PASS 2024-08-20T22:46:54.8202471Z fbnetc_100 PASS 2024-08-20T22:46:54.8207520Z fbnetv3_b PASS 2024-08-20T22:46:54.8212526Z gernet_l PASS 2024-08-20T22:46:54.8217541Z ghostnet_100 PASS 2024-08-20T22:46:54.8223125Z gluon_inception_v3 PASS 2024-08-20T22:46:54.8227857Z gmixer_24_224 PASS 2024-08-20T22:46:54.8232872Z gmlp_s16_224 PASS 2024-08-20T22:46:54.8237958Z hrnet_w18 PASS 2024-08-20T22:46:54.8243039Z inception_v3 PASS 2024-08-20T22:46:54.8247953Z jx_nest_base PASS 2024-08-20T22:46:54.8253079Z lcnet_050 XFAIL 2024-08-20T22:46:54.8257963Z levit_128 PASS 2024-08-20T22:46:54.8263407Z mixer_b16_224 PASS 2024-08-20T22:46:54.8699614Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/aot_eager_timm_training.csv 2024-08-20T22:46:55.1338302Z adv_inception_v3 PASS 2024-08-20T22:46:55.1342378Z beit_base_patch16_224 PASS 2024-08-20T22:46:55.1347148Z botnet26t_256 PASS 2024-08-20T22:46:55.1352288Z cait_m36_384 PASS 2024-08-20T22:46:55.1357165Z coat_lite_mini PASS 2024-08-20T22:46:55.1362152Z convit_base PASS 2024-08-20T22:46:55.1367089Z convmixer_768_32 PASS 2024-08-20T22:46:55.1371930Z convnext_base PASS 2024-08-20T22:46:55.1376859Z crossvit_9_240 PASS 2024-08-20T22:46:55.1382111Z cspdarknet53 PASS 2024-08-20T22:46:55.1386920Z deit_base_distilled_patch16_224 PASS 2024-08-20T22:46:55.1391856Z dla102 PASS 2024-08-20T22:46:55.1397413Z dm_nfnet_f0 PASS 2024-08-20T22:46:55.1402375Z dpn107 PASS 2024-08-20T22:46:55.1407327Z eca_botnext26ts_256 PASS 2024-08-20T22:46:55.1412379Z eca_halonext26ts PASS 2024-08-20T22:46:55.1417394Z ese_vovnet19b_dw PASS 2024-08-20T22:46:55.1422694Z fbnetc_100 PASS 2024-08-20T22:46:55.1427672Z fbnetv3_b PASS 2024-08-20T22:46:55.1432464Z gernet_l PASS 2024-08-20T22:46:55.1437593Z ghostnet_100 PASS 2024-08-20T22:46:55.1442437Z gluon_inception_v3 PASS 2024-08-20T22:46:55.1447442Z gmixer_24_224 PASS 2024-08-20T22:46:55.1452292Z gmlp_s16_224 PASS 2024-08-20T22:46:55.1457196Z hrnet_w18 PASS 2024-08-20T22:46:55.1462428Z inception_v3 PASS 2024-08-20T22:46:55.1467296Z jx_nest_base PASS 2024-08-20T22:46:55.1472187Z lcnet_050 PASS 2024-08-20T22:46:55.1477142Z levit_128 PASS 2024-08-20T22:46:55.1482121Z mixer_b16_224 PASS 2024-08-20T22:46:55.1923139Z + cleanup_workspace 2024-08-20T22:46:55.1924239Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T22:46:55.1925596Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T22:46:55.1926642Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T22:46:55.1927300Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T22:46:55.1928103Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T22:46:55.1929046Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T22:46:55.1929744Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T22:46:55.9378504Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T22:46:55.9379289Z cat test/**/*_toprint.log || true 2024-08-20T22:46:55.9392975Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:55.9394039Z env: 2024-08-20T22:46:55.9394340Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:55.9394798Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:55.9395526Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:55.9396167Z ##[endgroup] 2024-08-20T22:46:55.9493902Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T22:46:55.9526766Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T22:46:55.9527245Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T22:46:55.9535902Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:55.9536400Z env: 2024-08-20T22:46:55.9536687Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:55.9537141Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:55.9537879Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:55.9538529Z MONITOR_SCRIPT_PID: 52460 2024-08-20T22:46:55.9538887Z ##[endgroup] 2024-08-20T22:46:55.9740999Z Prepare all required actions 2024-08-20T22:46:55.9741481Z Getting action download info 2024-08-20T22:46:56.1999468Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T22:46:56.3852057Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T22:46:56.3852550Z with: 2024-08-20T22:46:56.3853091Z file-suffix: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476 2024-08-20T22:46:56.3853730Z s3-bucket: gha-artifacts 2024-08-20T22:46:56.3854079Z env: 2024-08-20T22:46:56.3854383Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:56.3854837Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:56.3855573Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:56.3856218Z ##[endgroup] 2024-08-20T22:46:56.3883227Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T22:46:56.3883881Z # Remove any previous test jsons if they exist 2024-08-20T22:46:56.3884573Z rm -f test-jsons-*.zip 2024-08-20T22:46:56.3885089Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T22:46:56.3895714Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:56.3896211Z env: 2024-08-20T22:46:56.3896491Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:56.3896935Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:56.3897665Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:56.3898517Z FILE_SUFFIX: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476 2024-08-20T22:46:56.3899225Z ##[endgroup] 2024-08-20T22:46:56.4236106Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T22:46:56.4266004Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T22:46:56.4267400Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T22:46:56.4272306Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T22:46:56.4277956Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T22:46:56.4737636Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T22:46:56.4738284Z # Remove any previous test reports if they exist 2024-08-20T22:46:56.4738811Z rm -f test-reports-*.zip 2024-08-20T22:46:56.4739642Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T22:46:56.4748846Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:56.4749343Z env: 2024-08-20T22:46:56.4749632Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:56.4750090Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:56.4750828Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:56.4751691Z FILE_SUFFIX: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476 2024-08-20T22:46:56.4752306Z ##[endgroup] 2024-08-20T22:46:56.5012983Z adding: test/test-reports/inference_timm_models.csv (deflated 68%) 2024-08-20T22:46:56.5013818Z adding: test/test-reports/training_timm_models.csv (deflated 66%) 2024-08-20T22:46:56.5017627Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-08-20T22:46:56.5018577Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 73%) 2024-08-20T22:46:56.5052402Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T22:46:56.5053261Z # Remove any previous usage logs if they exist 2024-08-20T22:46:56.5053787Z rm -f logs-*.zip 2024-08-20T22:46:56.5054462Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T22:46:56.5055258Z # so check to see if the file exists first 2024-08-20T22:46:56.5055762Z if [ -f 'usage_log.txt' ]; then 2024-08-20T22:46:56.5056297Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T22:46:56.5057030Z fi 2024-08-20T22:46:56.5057422Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T22:46:56.5057998Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T22:46:56.5058496Z fi 2024-08-20T22:46:56.5067422Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:56.5067933Z env: 2024-08-20T22:46:56.5068212Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:56.5068671Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:56.5069447Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:56.5070323Z FILE_SUFFIX: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476 2024-08-20T22:46:56.5070927Z ##[endgroup] 2024-08-20T22:46:56.5141945Z adding: usage_log.txt (deflated 92%) 2024-08-20T22:46:56.5232255Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T22:46:56.5232999Z # Remove any previous debugging artifacts if they exist 2024-08-20T22:46:56.5233748Z rm -f debug-*.zip 2024-08-20T22:46:56.5234142Z if [ -d 'test/debug' ]; then 2024-08-20T22:46:56.5234653Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T22:46:56.5235143Z fi 2024-08-20T22:46:56.5243778Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:56.5244282Z env: 2024-08-20T22:46:56.5244566Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:56.5245024Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:56.5245757Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:56.5246628Z FILE_SUFFIX: test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476 2024-08-20T22:46:56.5247234Z ##[endgroup] 2024-08-20T22:46:56.5343873Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:46:56.5344306Z with: 2024-08-20T22:46:56.5344590Z s3-bucket: gha-artifacts 2024-08-20T22:46:56.5345036Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:46:56.5345523Z retention-days: 14 2024-08-20T22:46:56.5345852Z if-no-files-found: warn 2024-08-20T22:46:56.5346209Z path: test-jsons-*.zip 2024-08-20T22:46:56.5346548Z name: artifact 2024-08-20T22:46:56.5346845Z region: us-east-1 2024-08-20T22:46:56.5347144Z env: 2024-08-20T22:46:56.5347420Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:56.5347869Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:56.5348596Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:56.5349231Z ##[endgroup] 2024-08-20T22:46:56.8909515Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:46:56.8910381Z With the provided path, there will be 1 file uploaded 2024-08-20T22:46:56.8911166Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:46:56.9711096Z Starting upload of test-jsons-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476.zip 2024-08-20T22:46:57.1557069Z Finished upload of test-jsons-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476.zip 2024-08-20T22:46:57.1841458Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:46:57.1842016Z with: 2024-08-20T22:46:57.1842416Z s3-bucket: gha-artifacts 2024-08-20T22:46:57.1842951Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:46:57.1843543Z retention-days: 14 2024-08-20T22:46:57.1844047Z if-no-files-found: error 2024-08-20T22:46:57.1844588Z path: test-reports-*.zip 2024-08-20T22:46:57.1845093Z name: artifact 2024-08-20T22:46:57.1845554Z region: us-east-1 2024-08-20T22:46:57.1845915Z env: 2024-08-20T22:46:57.1846306Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:57.1846890Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:57.1847738Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:57.1848433Z ##[endgroup] 2024-08-20T22:46:57.5418704Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:46:57.5419609Z With the provided path, there will be 1 file uploaded 2024-08-20T22:46:57.5420248Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:46:57.5475865Z Starting upload of test-reports-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476.zip 2024-08-20T22:46:57.6813471Z Finished upload of test-reports-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476.zip 2024-08-20T22:46:57.7103836Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:46:57.7104275Z with: 2024-08-20T22:46:57.7104553Z s3-bucket: gha-artifacts 2024-08-20T22:46:57.7104986Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:46:57.7105459Z retention-days: 14 2024-08-20T22:46:57.7105790Z if-no-files-found: ignore 2024-08-20T22:46:57.7106166Z path: logs-*.zip 2024-08-20T22:46:57.7106486Z name: artifact 2024-08-20T22:46:57.7106801Z region: us-east-1 2024-08-20T22:46:57.7107105Z env: 2024-08-20T22:46:57.7107399Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:57.7107874Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:57.7108814Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:57.7109448Z ##[endgroup] 2024-08-20T22:46:58.0647508Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:46:58.0648184Z With the provided path, there will be 1 file uploaded 2024-08-20T22:46:58.0648886Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:46:58.0704368Z Starting upload of logs-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476.zip 2024-08-20T22:46:58.1949226Z Finished upload of logs-test-aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026299476.zip 2024-08-20T22:46:58.2223388Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:46:58.2223831Z with: 2024-08-20T22:46:58.2224124Z s3-bucket: gha-artifacts 2024-08-20T22:46:58.2224563Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:46:58.2225059Z retention-days: 14 2024-08-20T22:46:58.2225411Z if-no-files-found: ignore 2024-08-20T22:46:58.2225782Z path: debug-*.zip 2024-08-20T22:46:58.2226104Z name: artifact 2024-08-20T22:46:58.2226411Z region: us-east-1 2024-08-20T22:46:58.2226717Z env: 2024-08-20T22:46:58.2227005Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:58.2227459Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:58.2228187Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:58.2228833Z ##[endgroup] 2024-08-20T22:46:58.5700583Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T22:46:58.5977475Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T22:46:58.5977974Z # shellcheck disable=SC2156 2024-08-20T22:46:58.5978784Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T22:46:58.5988186Z shell: /usr/bin/bash -e {0} 2024-08-20T22:46:58.5988546Z env: 2024-08-20T22:46:58.5988850Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:58.5989332Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:58.5990084Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:58.5990725Z ##[endgroup] 2024-08-20T22:46:58.8626809Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T22:46:58.8627382Z with: 2024-08-20T22:46:58.8627650Z env: 2024-08-20T22:46:58.8627934Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:58.8628389Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:58.8629125Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:58.8629767Z ##[endgroup] 2024-08-20T22:46:58.8648990Z ##[group]Run set -eou pipefail 2024-08-20T22:46:58.8649403Z set -eou pipefail 2024-08-20T22:46:58.8649753Z  2024-08-20T22:46:58.8650263Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T22:46:58.8650903Z for _ in $(seq 1440); do 2024-08-20T22:46:58.8651373Z  # Break if no ssh session exists anymore 2024-08-20T22:46:58.8651873Z  if [ "$(who)" = "" ]; then 2024-08-20T22:46:58.8652283Z  break 2024-08-20T22:46:58.8652619Z  fi 2024-08-20T22:46:58.8652914Z  echo "." 2024-08-20T22:46:58.8653244Z  sleep 5 2024-08-20T22:46:58.8653557Z done 2024-08-20T22:46:58.8662371Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:58.8662873Z env: 2024-08-20T22:46:58.8663156Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:58.8663615Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:58.8664350Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:58.8664987Z ##[endgroup] 2024-08-20T22:46:58.8697759Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T22:46:58.8786512Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:46:58.8787290Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:46:58.8787889Z # shellcheck disable=SC2046 2024-08-20T22:46:58.8788372Z docker stop $(docker ps -q) || true 2024-08-20T22:46:58.8788856Z # Prune all of the docker images 2024-08-20T22:46:58.8789316Z docker system prune -af 2024-08-20T22:46:58.8798645Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:46:58.8799150Z env: 2024-08-20T22:46:58.8799427Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:46:58.8799884Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:46:58.8800614Z DOCKER_CONTAINER_ID: dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:46:58.8801256Z ##[endgroup] 2024-08-20T22:46:59.3565222Z dd0d6f041c6d 2024-08-20T22:47:00.7979162Z Deleted Containers: 2024-08-20T22:47:00.7979751Z dd0d6f041c6dbb200298a6c5196196c00ae76b958f73d8313c8f4c6b28236a67 2024-08-20T22:47:00.7980267Z 2024-08-20T22:47:11.7951259Z Deleted Images: 2024-08-20T22:47:11.7953293Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:47:11.7955901Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T22:47:11.7957540Z deleted: sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb 2024-08-20T22:47:11.7958394Z deleted: sha256:c2ce10014dd98f6abbfbf2265f9780a8204c2a7a27463e61ee565573a23d141d 2024-08-20T22:47:11.7959249Z deleted: sha256:ae249c0e6e6013bd6ffea2ca0b0160c015c9ebf7c03070bc2f983a9ff2415bf5 2024-08-20T22:47:11.7960070Z deleted: sha256:8a73883c1067c0e29ff128f553518682b47ed9707d2f35e7d5138e2caf029d61 2024-08-20T22:47:11.7960957Z deleted: sha256:9c040db729fa084584aaf5573082408a2196bc64e5952270c79dc610ed4680b6 2024-08-20T22:47:11.7961768Z deleted: sha256:17311eeeb8b71b16699880c0f646e185107496c4d41da5d23370603768ec030c 2024-08-20T22:47:11.7962589Z deleted: sha256:07cc10bece8501b1c8035d4385ad9e9867dd6e92fa32218a4dcb3e42af7e0411 2024-08-20T22:47:11.7963731Z deleted: sha256:f3e32d05099d1a3115a4722bc2e94a48752c3e965446773673808252671bb967 2024-08-20T22:47:11.7964578Z deleted: sha256:be7c3f41475e3e72ffe3ac2f92ab2f4cf49d5496b2aae738942ac3bb687d742c 2024-08-20T22:47:11.7965520Z deleted: sha256:4fcac490be825129b468cf458089f1fcf8fd56afc44ef4ab27c2eff41ab1ac90 2024-08-20T22:47:11.7966343Z deleted: sha256:31c66d80a2a4792897f267d363c0cf84fb821c5d4a275ab1636f778fd0284c08 2024-08-20T22:47:11.7967157Z deleted: sha256:8a8bcb6085e7e438e5533e8a5613e2ae5888543853568d023af3e2623a1593a9 2024-08-20T22:47:11.7967972Z deleted: sha256:2f6cb60029c79d11567b738f5bdbfb9cb1838cc0f194c4a5e6ae685178d2ed34 2024-08-20T22:47:11.7968786Z deleted: sha256:e546391804d3313f558e1ac16e93cb29a417a92e4eccaf0d4bd12af3bbcf39e3 2024-08-20T22:47:11.7969608Z deleted: sha256:1623b2aa16f2bdc6b572f6799498d9a92fa8114d07c440599b17aea72e22f8b1 2024-08-20T22:47:11.7970436Z deleted: sha256:c0386ca58b7cb7afee015b7e58d2f70525eaf73d6da2cc63868b20e14b1f9497 2024-08-20T22:47:11.7971271Z deleted: sha256:dcbf534ff289186e246531925fed90f72516eb3d63663bad234d611cdc025d62 2024-08-20T22:47:11.7972089Z deleted: sha256:0bab537af579f56c9cd51c7073fd53635fffb424b363236e0de38a7dd0e60ae8 2024-08-20T22:47:11.7972912Z deleted: sha256:d85a852ae1ee027e2fc17240ed68b87957b0a6d210c9119752d8979a0dcd55cf 2024-08-20T22:47:11.7973777Z deleted: sha256:f3821d761b555362e44cf9a1447d8fd2d95b37d54aea67c7519cd9e5db0c4d7e 2024-08-20T22:47:11.7974675Z deleted: sha256:c05e84a545db5dfcb0847c73d2b0f4749fa67aabce05705632ff7f76c84d4688 2024-08-20T22:47:11.7975512Z deleted: sha256:27054eae7ca67b2bd5ca0fd3f4488bf5c5f78e95487d8c45616f3d753d48a8b5 2024-08-20T22:47:11.7976339Z deleted: sha256:9804139f6ae97796aaa993fe4b89ea0cf111eb4d30a79f4c87c339c2e1e04fa0 2024-08-20T22:47:11.7977368Z deleted: sha256:ad6ae2fc594aaa359ff4e287323836570454eb24dd162930655e414d1d77c6dd 2024-08-20T22:47:11.7978197Z deleted: sha256:9cfbb36e328af52a248853a46aab88e266c09e2b44a03e0a42194a031b9b9453 2024-08-20T22:47:11.7979248Z deleted: sha256:a8a842bd7f37b8d50b6d12d6f95f46bbba57953db466fcbde9ba522625f024ef 2024-08-20T22:47:11.7980085Z deleted: sha256:da9738654b3c86f89ebeed2560f1bae093f1219f1eff90b0394b7f211ecabd04 2024-08-20T22:47:11.7980916Z deleted: sha256:cc683bf0cb245394e0f568984b2dd4fce1b6a3a7fe7c44734bae1aa1caaa3251 2024-08-20T22:47:11.7981744Z deleted: sha256:965f220e71c7d1ec19aaf41b1529ab287beb9d86eb8482c4c14094485ab34475 2024-08-20T22:47:11.7982569Z deleted: sha256:da32ecec41a73ce7dd29464f90c7951510a18cd378753659a78cd7fabd10dc36 2024-08-20T22:47:11.7983383Z deleted: sha256:d828d187cee9ad48dbc3702fb61d359619dd8a1ed582878680b51686bf117e5a 2024-08-20T22:47:11.7984198Z deleted: sha256:712111f4b12ff8e2787597d5fdcb043bd0824e55ddd97f262ef550506ca195f0 2024-08-20T22:47:11.7985011Z deleted: sha256:a064eea405958ad14dc6c794e5df9fd1d4515424272801ec559d45a59317309a 2024-08-20T22:47:11.7985837Z deleted: sha256:e124382d136bd5e9a8b409ca79c1f1ef76458deaf444b1b2060d3fa99dbeb506 2024-08-20T22:47:11.7986654Z deleted: sha256:6e10fe757cc190a1b31b47010f4f7610e02cb1b4b438f190dcd301aa4fe2278e 2024-08-20T22:47:11.7987470Z deleted: sha256:4cea352e10f46ca4729f96ac6c4600fc53d4c36e7658c2e0f19f4f309cd4a2a7 2024-08-20T22:47:11.7988287Z deleted: sha256:518bba00fe1d0933053066d21cac0235c9fb485abbc41557ec184b293095c924 2024-08-20T22:47:11.7989113Z deleted: sha256:a64f9ca42e9f27ff2690c15a5c84312e6efaccad45668d9b496cb542165cfb5a 2024-08-20T22:47:11.7989940Z deleted: sha256:096ea0fa14a6c420658f5276b8a8f5d264c0de7e18161ceda2cfc2f4a92cd9bf 2024-08-20T22:47:11.7990770Z deleted: sha256:6f91c94ce6e3f8034c91c66ee25c66a3d85dba9d813b749bd6d2000942a99edc 2024-08-20T22:47:11.7991612Z deleted: sha256:8f8d255eeae546cead8f9a94d237fa57a99940c0f5d51bc29a9d41c7df9cffd9 2024-08-20T22:47:11.7992459Z deleted: sha256:fe05276f4a49f91eef56bfbc3da87cfff5eb116a827fa2d30efe42ce1c134b40 2024-08-20T22:47:11.7993600Z deleted: sha256:5f83cd75d0669c813d4f58978c92e14305cd8f6a07eb9a60665a583460fa7b59 2024-08-20T22:47:11.7994412Z deleted: sha256:8f7ab6366169d39d8953129f650f1927d2fb1328b3f11e2fd9076adb5f46b93a 2024-08-20T22:47:11.7995414Z deleted: sha256:4146cfd5dba0f5308954749f98596b14a1d8f6d23bd93f7ad86c6c3d72239328 2024-08-20T22:47:11.7996259Z deleted: sha256:c6d83cb9ccc606ff63bb3c4d5a5e1ff1450c619b5aae494303a94ae25971cca0 2024-08-20T22:47:11.7997077Z deleted: sha256:71e037e0a3e0963c4565534a7387c082afcb04475d3ac629acf7a0ab2f49fadc 2024-08-20T22:47:11.7997869Z deleted: sha256:4b31135c564686bdaf23262b901237880aea635655721cc7e8c95dc42b05e06d 2024-08-20T22:47:11.7998840Z deleted: sha256:d09bd5c2f35cedcb846543941ee64ce1f684b52e4285696c93b3fb2032da8579 2024-08-20T22:47:11.7999929Z deleted: sha256:4de7fb7d4fc22212aae5fc1ceb637526916d0cced240d7c93d5edd6b0ea28013 2024-08-20T22:47:11.8001009Z deleted: sha256:69890e3e82746e10aff2f1084f11afb4b95a4a6c26b30f64b7001bc0a5fe32bf 2024-08-20T22:47:11.8002069Z deleted: sha256:0156df53b3224e645336ea52b4bb816f1e8d8c9b591748cd645e5e5649fa507b 2024-08-20T22:47:11.8003200Z deleted: sha256:0664624cb9cc4a386f31bf49919a8d8a11d9fea428d05585d9e6228f754bc7cd 2024-08-20T22:47:11.8004046Z deleted: sha256:8c2ddd5fa535c768c058a23f5cc4b6c64cdd71bb7458491282aabc7f7bb94d69 2024-08-20T22:47:11.8004876Z deleted: sha256:8dd03dbe0e1c5f47dd333074dff4bd00415f2f63727a65bf17866774e2ebbd0d 2024-08-20T22:47:11.8005695Z deleted: sha256:3c56b63e094bd18fad9da7a541d537210bca3b730c590d3641213259f6908d6f 2024-08-20T22:47:11.8006506Z deleted: sha256:51484fef07bd1266d4d6ff3d767751172660fc4cc6a1ffe883523a63c985e31e 2024-08-20T22:47:11.8007311Z deleted: sha256:03e83c94023982ab7dd8dc65a6df2024e342f1b760c91ec7a680ee45824869c7 2024-08-20T22:47:11.8008113Z deleted: sha256:58440ab3e8175b5de94a76485c8689a6dc186015475ebf4b386ebba309c5da34 2024-08-20T22:47:11.8008917Z deleted: sha256:9a980f3383e8aa511f0ef68982cb6dfc53b38d1ac8c3d673cb3ba3d45158c91e 2024-08-20T22:47:11.8009907Z deleted: sha256:cb0269c9cf1c47b6c4188ca7be4db261b5d3fc4b4a5153629e222dba7b1f9f98 2024-08-20T22:47:11.8010739Z deleted: sha256:41f1ba870cdffc05e0bf8d87b640fb9d4c153d7f4f3b775f37f94e9f3d584aad 2024-08-20T22:47:11.8011574Z deleted: sha256:7b502ab352f48e8b6eb5e6824a29f308952a5ee7c7d987562e1df7e47639d7f1 2024-08-20T22:47:11.8012385Z deleted: sha256:6ba5217e18738660e6ca8b7a7ec88845e70b0fd781301520bf46c71d99df3144 2024-08-20T22:47:11.8013249Z deleted: sha256:dd5a1c233594780163726bdbbf5de00f7b6b8dca1dace8a001340714c54baedb 2024-08-20T22:47:11.8014063Z deleted: sha256:c27f0db27da73e01f67f6cb78d10f355e350d829479ba4351d82055bb29f6597 2024-08-20T22:47:11.8014880Z deleted: sha256:4e3ce3c854f35c08be3dd95e0d6e718512a885e4614291d7d8b7cd23f48b7dd8 2024-08-20T22:47:11.8015703Z deleted: sha256:c2d6fb088bbec16e60cf0ad74b016f7287c92e659ec2db01271bc309b4dc6d83 2024-08-20T22:47:11.8016530Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-08-20T22:47:11.8017337Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-08-20T22:47:11.8018143Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-08-20T22:47:11.8018960Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-08-20T22:47:11.8019945Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-08-20T22:47:11.8020755Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-08-20T22:47:11.8021581Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-08-20T22:47:11.8022408Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-08-20T22:47:11.8023240Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-08-20T22:47:11.8024062Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-08-20T22:47:11.8024872Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-20T22:47:11.8025371Z 2024-08-20T22:47:11.8025521Z Total reclaimed space: 44.92GB 2024-08-20T22:47:11.8093735Z Post job cleanup. 2024-08-20T22:47:11.8145159Z Post job cleanup. 2024-08-20T22:47:11.9024225Z [command]/usr/bin/git version 2024-08-20T22:47:11.9072593Z git version 2.40.1 2024-08-20T22:47:11.9111456Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/d7257953-7697-4d2e-8984-ddd99bee091b' before making global git config changes 2024-08-20T22:47:11.9112705Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:47:11.9116501Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:47:11.9164404Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:47:11.9208879Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:47:11.9577770Z Entering 'android/libs/fbjni' 2024-08-20T22:47:11.9645835Z Entering 'third_party/FP16' 2024-08-20T22:47:11.9715582Z Entering 'third_party/FXdiv' 2024-08-20T22:47:11.9784889Z Entering 'third_party/NNPACK' 2024-08-20T22:47:11.9853216Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:47:11.9921780Z Entering 'third_party/XNNPACK' 2024-08-20T22:47:12.0007070Z Entering 'third_party/benchmark' 2024-08-20T22:47:12.0075431Z Entering 'third_party/cpp-httplib' 2024-08-20T22:47:12.0144231Z Entering 'third_party/cpuinfo' 2024-08-20T22:47:12.0212352Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:47:12.0284384Z Entering 'third_party/cutlass' 2024-08-20T22:47:12.0360941Z Entering 'third_party/eigen' 2024-08-20T22:47:12.0432099Z Entering 'third_party/fbgemm' 2024-08-20T22:47:12.0501277Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:47:12.0569284Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:47:12.0638299Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:47:12.0713506Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:47:12.0782976Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:47:12.0855383Z Entering 'third_party/flatbuffers' 2024-08-20T22:47:12.0926879Z Entering 'third_party/fmt' 2024-08-20T22:47:12.0996052Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:47:12.1065608Z Entering 'third_party/gloo' 2024-08-20T22:47:12.1135143Z Entering 'third_party/googletest' 2024-08-20T22:47:12.1203876Z Entering 'third_party/ideep' 2024-08-20T22:47:12.1269813Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:47:12.1344871Z Entering 'third_party/ittapi' 2024-08-20T22:47:12.1413696Z Entering 'third_party/kineto' 2024-08-20T22:47:12.1481165Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:47:12.1552858Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:47:12.1622732Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:47:12.1691080Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:47:12.1760875Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:47:12.1827252Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:47:12.1899283Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:47:12.1967297Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:47:12.2034906Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:47:12.2105602Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:47:12.2176149Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:47:12.2246088Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:47:12.2318783Z Entering 'third_party/mimalloc' 2024-08-20T22:47:12.2386926Z Entering 'third_party/nccl/nccl' 2024-08-20T22:47:12.2456375Z Entering 'third_party/nlohmann' 2024-08-20T22:47:12.2526778Z Entering 'third_party/onnx' 2024-08-20T22:47:12.2610442Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:47:12.2678302Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:47:12.2751485Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:47:12.2821324Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:47:12.2890915Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:47:12.2958285Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:47:12.3024926Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:47:12.3094825Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:47:12.3165283Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:47:12.3231559Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:47:12.3296245Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:47:12.3363412Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:47:12.3434588Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:47:12.3523128Z Entering 'third_party/pocketfft' 2024-08-20T22:47:12.3592418Z Entering 'third_party/protobuf' 2024-08-20T22:47:12.3662783Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:47:12.3734864Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:47:12.3807040Z Entering 'third_party/psimd' 2024-08-20T22:47:12.3876129Z Entering 'third_party/pthreadpool' 2024-08-20T22:47:12.3948066Z Entering 'third_party/pybind11' 2024-08-20T22:47:12.4016409Z Entering 'third_party/python-peachpy' 2024-08-20T22:47:12.4085233Z Entering 'third_party/sleef' 2024-08-20T22:47:12.4153309Z Entering 'third_party/tensorpipe' 2024-08-20T22:47:12.4220731Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:47:12.4286081Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:47:12.4351253Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:47:12.4417878Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:47:12.4481258Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:47:12.4580913Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:47:12.4617225Z http.https://github.com/.extraheader 2024-08-20T22:47:12.4626468Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T22:47:12.4672062Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:47:12.5045914Z Entering 'android/libs/fbjni' 2024-08-20T22:47:12.5088848Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5134100Z Entering 'third_party/FP16' 2024-08-20T22:47:12.5178695Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5222863Z Entering 'third_party/FXdiv' 2024-08-20T22:47:12.5267520Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5311612Z Entering 'third_party/NNPACK' 2024-08-20T22:47:12.5356355Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5401133Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:47:12.5445585Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5489702Z Entering 'third_party/XNNPACK' 2024-08-20T22:47:12.5535461Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5595582Z Entering 'third_party/benchmark' 2024-08-20T22:47:12.5640652Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5685098Z Entering 'third_party/cpp-httplib' 2024-08-20T22:47:12.5730203Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5774367Z Entering 'third_party/cpuinfo' 2024-08-20T22:47:12.5819455Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5863896Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:47:12.5908619Z http.https://github.com/.extraheader 2024-08-20T22:47:12.5952750Z Entering 'third_party/cutlass' 2024-08-20T22:47:12.5999462Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6052425Z Entering 'third_party/eigen' 2024-08-20T22:47:12.6096008Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6142490Z Entering 'third_party/fbgemm' 2024-08-20T22:47:12.6186364Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6231751Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:47:12.6275509Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6318990Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:47:12.6361711Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6405181Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:47:12.6448277Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6501881Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:47:12.6545061Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6587538Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:47:12.6631601Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6678507Z Entering 'third_party/flatbuffers' 2024-08-20T22:47:12.6724669Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6770752Z Entering 'third_party/fmt' 2024-08-20T22:47:12.6815256Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6858948Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:47:12.6903927Z http.https://github.com/.extraheader 2024-08-20T22:47:12.6948242Z Entering 'third_party/gloo' 2024-08-20T22:47:12.6993001Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7040651Z Entering 'third_party/googletest' 2024-08-20T22:47:12.7085239Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7129877Z Entering 'third_party/ideep' 2024-08-20T22:47:12.7174543Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7217083Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:47:12.7259358Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7310952Z Entering 'third_party/ittapi' 2024-08-20T22:47:12.7354868Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7398449Z Entering 'third_party/kineto' 2024-08-20T22:47:12.7442569Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7485518Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:47:12.7528643Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7571907Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:47:12.7617628Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7663816Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:47:12.7707421Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7752035Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:47:12.7795745Z http.https://github.com/.extraheader 2024-08-20T22:47:12.7840933Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:47:12.7884693Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8020440Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:47:12.8063607Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8110117Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:47:12.8152427Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8200760Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:47:12.8242895Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8287211Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:47:12.8330325Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8375742Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:47:12.8418594Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8465250Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:47:12.8508660Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8550866Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:47:12.8591698Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8638015Z Entering 'third_party/mimalloc' 2024-08-20T22:47:12.8681597Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8725452Z Entering 'third_party/nccl/nccl' 2024-08-20T22:47:12.8769290Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8819443Z Entering 'third_party/nlohmann' 2024-08-20T22:47:12.8862403Z http.https://github.com/.extraheader 2024-08-20T22:47:12.8906858Z Entering 'third_party/onnx' 2024-08-20T22:47:12.8949986Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9008242Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:47:12.9053840Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9095585Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:47:12.9137353Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9183864Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:47:12.9230788Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9276404Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:47:12.9318861Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9361784Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:47:12.9404973Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9448637Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:47:12.9490606Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9534539Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:47:12.9576392Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9619727Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:47:12.9663508Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9710085Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:47:12.9752943Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9796802Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:47:12.9839706Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9881065Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:47:12.9924924Z http.https://github.com/.extraheader 2024-08-20T22:47:12.9970515Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:47:13.0014220Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0063041Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:47:13.0107506Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0173087Z Entering 'third_party/pocketfft' 2024-08-20T22:47:13.0217615Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0261874Z Entering 'third_party/protobuf' 2024-08-20T22:47:13.0306604Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0352688Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:47:13.0397550Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0440373Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:47:13.0483360Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0531523Z Entering 'third_party/psimd' 2024-08-20T22:47:13.0577082Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0622352Z Entering 'third_party/pthreadpool' 2024-08-20T22:47:13.0666206Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0710391Z Entering 'third_party/pybind11' 2024-08-20T22:47:13.0754738Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0799632Z Entering 'third_party/python-peachpy' 2024-08-20T22:47:13.0844298Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0888642Z Entering 'third_party/sleef' 2024-08-20T22:47:13.0933720Z http.https://github.com/.extraheader 2024-08-20T22:47:13.0978783Z Entering 'third_party/tensorpipe' 2024-08-20T22:47:13.1023616Z http.https://github.com/.extraheader 2024-08-20T22:47:13.1066604Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:47:13.1112416Z http.https://github.com/.extraheader 2024-08-20T22:47:13.1156294Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:47:13.1198832Z http.https://github.com/.extraheader 2024-08-20T22:47:13.1241615Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:47:13.1284125Z http.https://github.com/.extraheader 2024-08-20T22:47:13.1328066Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:47:13.1370192Z http.https://github.com/.extraheader 2024-08-20T22:47:13.1412150Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:47:13.1455809Z http.https://github.com/.extraheader 2024-08-20T22:47:13.1616026Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T22:47:13.1636149Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T22:47:13.1644173Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:47:13.1644687Z ##[endgroup] 2024-08-20T22:47:20.0606013Z Cleaning up orphan processes