2024-08-20T22:04:44.4903951Z Current runner version: '2.319.1' 2024-08-20T22:04:44.4910244Z Runner name: 'i-00e21703921bea6d0' 2024-08-20T22:04:44.4911098Z Runner group name: 'Default' 2024-08-20T22:04:44.4912017Z Machine name: 'ip-10-0-46-207' 2024-08-20T22:04:44.4928505Z Testing runner upgrade compatibility 2024-08-20T22:04:44.6284643Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T22:04:44.6287182Z Actions: read 2024-08-20T22:04:44.6287721Z Attestations: read 2024-08-20T22:04:44.6288318Z Checks: read 2024-08-20T22:04:44.6288817Z Contents: read 2024-08-20T22:04:44.6289355Z Deployments: read 2024-08-20T22:04:44.6289941Z Discussions: read 2024-08-20T22:04:44.6290418Z Issues: read 2024-08-20T22:04:44.6290930Z Metadata: read 2024-08-20T22:04:44.6291500Z Packages: read 2024-08-20T22:04:44.6291967Z Pages: read 2024-08-20T22:04:44.6292479Z PullRequests: read 2024-08-20T22:04:44.6293090Z RepositoryProjects: read 2024-08-20T22:04:44.6293690Z SecurityEvents: read 2024-08-20T22:04:44.6294219Z Statuses: read 2024-08-20T22:04:44.6294999Z ##[endgroup] 2024-08-20T22:04:44.6298372Z Secret source: Actions 2024-08-20T22:04:44.6299272Z Prepare workflow directory 2024-08-20T22:04:44.7714216Z Prepare all required actions 2024-08-20T22:04:44.7905007Z Getting action download info 2024-08-20T22:04:44.9846785Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T22:04:45.5402057Z Download action repository 'pytorch/pytorch@main' (SHA:1ae5d5bb62141d7c9b1b0b66c66a462b4e10b1f2) 2024-08-20T22:04:49.5939770Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T22:04:49.7775425Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T22:04:50.0601062Z Getting action download info 2024-08-20T22:04:50.1564849Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T22:04:50.3288896Z Getting action download info 2024-08-20T22:04:50.4072875Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T22:04:50.5481725Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T22:04:50.5483891Z ##[group] Inputs 2024-08-20T22:04:50.5484368Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:04:50.5491371Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:04:50.5499138Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:04:50.5500304Z sync-tag: 2024-08-20T22:04:50.5501179Z timeout-minutes: 240 2024-08-20T22:04:50.5501522Z use-gha: 2024-08-20T22:04:50.5501818Z dashboard-tag: 2024-08-20T22:04:50.5502149Z s3-bucket: gha-artifacts 2024-08-20T22:04:50.5502624Z aws-role-to-assume: 2024-08-20T22:04:50.5502985Z ##[endgroup] 2024-08-20T22:04:50.5504046Z Complete job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:04:50.6194635Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T22:04:50.6348830Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T22:04:50.6360567Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:04:50.6361110Z ##[endgroup] 2024-08-20T22:04:53.4844705Z Runner Type: linux.g5.4xlarge.nvidia.gpu 2024-08-20T22:04:53.4845248Z Instance Type: g5.4xlarge 2024-08-20T22:04:53.4845969Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T22:04:53.4846534Z AMI ID: ami-06c68f701d8090592 2024-08-20T22:04:59.2967115Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T22:04:59.2967675Z with: 2024-08-20T22:04:59.2968385Z github-secret: *** 2024-08-20T22:04:59.2969346Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:04:59.2970344Z activate-with-label: false 2024-08-20T22:04:59.2970720Z label: with-ssh 2024-08-20T22:04:59.2971053Z remove-existing-keys: true 2024-08-20T22:04:59.2971424Z fail-silently: true 2024-08-20T22:04:59.2971751Z env: 2024-08-20T22:04:59.2972036Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:04:59.2972412Z ##[endgroup] 2024-08-20T22:04:59.3881164Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T22:04:59.3884022Z ciflow reference detected, attempting to extract PR number 2024-08-20T22:04:59.6992004Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T22:04:59.7982708Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T22:04:59.7983351Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T22:04:59.8733341Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T22:04:59.8747481Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T22:04:59.8771222Z Login using: ssh ec2-user@ec2-44-202-154-169.compute-1.amazonaws.com 2024-08-20T22:04:59.8772021Z All testing is done inside the container, to start an interactive session run: 2024-08-20T22:04:59.8772848Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:04:59.8905753Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T22:04:59.8906314Z with: 2024-08-20T22:04:59.8906601Z submodules: recursive 2024-08-20T22:04:59.8906937Z fetch-depth: 0 2024-08-20T22:04:59.8907234Z env: 2024-08-20T22:04:59.8907511Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:04:59.8907855Z ##[endgroup] 2024-08-20T22:04:59.9110629Z ##[group]Run retry () { 2024-08-20T22:04:59.9111020Z retry () { 2024-08-20T22:04:59.9111541Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T22:04:59.9112098Z } 2024-08-20T22:04:59.9112410Z echo "${GITHUB_WORKSPACE}" 2024-08-20T22:04:59.9112916Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T22:04:59.9113494Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:04:59.9113964Z else 2024-08-20T22:04:59.9114312Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:04:59.9114741Z fi 2024-08-20T22:04:59.9115342Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T22:04:59.9125183Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:04:59.9125698Z env: 2024-08-20T22:04:59.9125985Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:04:59.9126329Z NO_SUDO: 2024-08-20T22:04:59.9126613Z ##[endgroup] 2024-08-20T22:04:59.9158665Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:05:04.6668336Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T22:05:04.6668776Z with: 2024-08-20T22:05:04.6669111Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:05:04.6669565Z fetch-depth: 0 2024-08-20T22:05:04.6669881Z submodules: recursive 2024-08-20T22:05:04.6670220Z quiet-checkout: true 2024-08-20T22:05:04.6670574Z repository: pytorch/pytorch 2024-08-20T22:05:04.6671062Z token: *** 2024-08-20T22:05:04.6671364Z ssh-strict: true 2024-08-20T22:05:04.6671690Z persist-credentials: true 2024-08-20T22:05:04.6672058Z clean: true 2024-08-20T22:05:04.6672406Z sparse-checkout-cone-mode: true 2024-08-20T22:05:04.6672795Z lfs: false 2024-08-20T22:05:04.6673090Z set-safe-directory: true 2024-08-20T22:05:04.6673430Z env: 2024-08-20T22:05:04.6673701Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:05:04.6674034Z ##[endgroup] 2024-08-20T22:05:04.7695058Z Syncing repository: pytorch/pytorch 2024-08-20T22:05:04.7697300Z ##[group]Getting Git version info 2024-08-20T22:05:04.7698467Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:05:04.7699651Z [command]/usr/bin/git version 2024-08-20T22:05:04.7700186Z git version 2.40.1 2024-08-20T22:05:04.7702008Z ##[endgroup] 2024-08-20T22:05:04.7714307Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/08fced67-8d72-423d-9cc5-e1122666c98c/.gitconfig' 2024-08-20T22:05:04.7717388Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/08fced67-8d72-423d-9cc5-e1122666c98c' before making global git config changes 2024-08-20T22:05:04.7718637Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:05:04.7722026Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:05:04.7779561Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:05:04.7783660Z ##[group]Initializing the repository 2024-08-20T22:05:04.7786775Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:05:04.7832708Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T22:05:04.7833892Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T22:05:04.7835009Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T22:05:04.7836101Z hint: 2024-08-20T22:05:04.7837065Z hint: git config --global init.defaultBranch 2024-08-20T22:05:04.7837703Z hint: 2024-08-20T22:05:04.7838793Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T22:05:04.7839964Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T22:05:04.7840744Z hint: 2024-08-20T22:05:04.7841204Z hint: git branch -m 2024-08-20T22:05:04.7842029Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T22:05:04.7848108Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T22:05:04.7894965Z ##[endgroup] 2024-08-20T22:05:04.7895606Z ##[group]Disabling automatic garbage collection 2024-08-20T22:05:04.7897840Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T22:05:04.7939411Z ##[endgroup] 2024-08-20T22:05:04.7939963Z ##[group]Setting up auth 2024-08-20T22:05:04.7945157Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:05:04.7984123Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:05:04.8330378Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:05:04.8372619Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:05:04.8875225Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:05:04.8931601Z ##[endgroup] 2024-08-20T22:05:04.8932431Z ##[group]Fetching the repository 2024-08-20T22:05:04.8942336Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T22:05:07.8271220Z remote: Enumerating objects: 1008452 2024-08-20T22:05:07.8271801Z remote: Enumerating objects: 1010208, done. 2024-08-20T22:05:07.8273454Z remote: Counting objects: 0% (1/1756) 2024-08-20T22:05:07.8273971Z remote: Counting objects: 1% (18/1756) 2024-08-20T22:05:07.8276208Z remote: Counting objects: 2% (36/1756) 2024-08-20T22:05:07.8276720Z remote: Counting objects: 3% (53/1756) 2024-08-20T22:05:07.8277221Z remote: Counting objects: 4% (71/1756) 2024-08-20T22:05:07.8277715Z remote: Counting objects: 5% (88/1756) 2024-08-20T22:05:07.8278217Z remote: Counting objects: 6% (106/1756) 2024-08-20T22:05:07.8280672Z remote: Counting objects: 7% (123/1756) 2024-08-20T22:05:07.8281202Z remote: Counting objects: 8% (141/1756) 2024-08-20T22:05:07.8281728Z remote: Counting objects: 9% (159/1756) 2024-08-20T22:05:07.8282374Z remote: Counting objects: 10% (176/1756) 2024-08-20T22:05:07.8283042Z remote: Counting objects: 11% (194/1756) 2024-08-20T22:05:07.8283707Z remote: Counting objects: 12% (211/1756) 2024-08-20T22:05:07.8284216Z remote: Counting objects: 13% (229/1756) 2024-08-20T22:05:07.8284807Z remote: Counting objects: 14% (246/1756) 2024-08-20T22:05:07.8285409Z remote: Counting objects: 15% (264/1756) 2024-08-20T22:05:07.8285980Z remote: Counting objects: 16% (281/1756) 2024-08-20T22:05:07.8286521Z remote: Counting objects: 17% (299/1756) 2024-08-20T22:05:07.8287020Z remote: Counting objects: 18% (317/1756) 2024-08-20T22:05:07.8287594Z remote: Counting objects: 19% (334/1756) 2024-08-20T22:05:07.8288230Z remote: Counting objects: 20% (352/1756) 2024-08-20T22:05:07.8288735Z remote: Counting objects: 21% (369/1756) 2024-08-20T22:05:07.8289323Z remote: Counting objects: 22% (387/1756) 2024-08-20T22:05:07.8289825Z remote: Counting objects: 23% (404/1756) 2024-08-20T22:05:07.8290450Z remote: Counting objects: 24% (422/1756) 2024-08-20T22:05:07.8291125Z remote: Counting objects: 25% (439/1756) 2024-08-20T22:05:07.8291966Z remote: Counting objects: 26% (457/1756) 2024-08-20T22:05:07.8292651Z remote: Counting objects: 27% (475/1756) 2024-08-20T22:05:07.8293183Z remote: Counting objects: 28% (492/1756) 2024-08-20T22:05:07.8293686Z remote: Counting objects: 29% (510/1756) 2024-08-20T22:05:07.8294325Z remote: Counting objects: 30% (527/1756) 2024-08-20T22:05:07.8294975Z remote: Counting objects: 31% (545/1756) 2024-08-20T22:05:07.8295615Z remote: Counting objects: 32% (562/1756) 2024-08-20T22:05:07.8296256Z remote: Counting objects: 33% (580/1756) 2024-08-20T22:05:07.8296890Z remote: Counting objects: 34% (598/1756) 2024-08-20T22:05:07.8297531Z remote: Counting objects: 35% (615/1756) 2024-08-20T22:05:07.8298170Z remote: Counting objects: 36% (633/1756) 2024-08-20T22:05:07.8298848Z remote: Counting objects: 37% (650/1756) 2024-08-20T22:05:07.8299496Z remote: Counting objects: 38% (668/1756) 2024-08-20T22:05:07.8300220Z remote: Counting objects: 39% (685/1756) 2024-08-20T22:05:07.8300727Z remote: Counting objects: 40% (703/1756) 2024-08-20T22:05:07.8301231Z remote: Counting objects: 41% (720/1756) 2024-08-20T22:05:07.8301733Z remote: Counting objects: 42% (738/1756) 2024-08-20T22:05:07.8302230Z remote: Counting objects: 43% (756/1756) 2024-08-20T22:05:07.8302896Z remote: Counting objects: 44% (773/1756) 2024-08-20T22:05:07.8303402Z remote: Counting objects: 45% (791/1756) 2024-08-20T22:05:07.8303906Z remote: Counting objects: 46% (808/1756) 2024-08-20T22:05:07.8304399Z remote: Counting objects: 47% (826/1756) 2024-08-20T22:05:07.8304908Z remote: Counting objects: 48% (843/1756) 2024-08-20T22:05:07.8305410Z remote: Counting objects: 49% (861/1756) 2024-08-20T22:05:07.8305911Z remote: Counting objects: 50% (878/1756) 2024-08-20T22:05:07.8306413Z remote: Counting objects: 51% (896/1756) 2024-08-20T22:05:07.8306926Z remote: Counting objects: 52% (914/1756) 2024-08-20T22:05:07.8307459Z remote: Counting objects: 53% (931/1756) 2024-08-20T22:05:07.8307981Z remote: Counting objects: 54% (949/1756) 2024-08-20T22:05:07.8308485Z remote: Counting objects: 55% (966/1756) 2024-08-20T22:05:07.8308988Z remote: Counting objects: 56% (984/1756) 2024-08-20T22:05:07.8309510Z remote: Counting objects: 57% (1001/1756) 2024-08-20T22:05:07.8310043Z remote: Counting objects: 58% (1019/1756) 2024-08-20T22:05:07.8310569Z remote: Counting objects: 59% (1037/1756) 2024-08-20T22:05:07.8311092Z remote: Counting objects: 60% (1054/1756) 2024-08-20T22:05:07.8311617Z remote: Counting objects: 61% (1072/1756) 2024-08-20T22:05:07.8312139Z remote: Counting objects: 62% (1089/1756) 2024-08-20T22:05:07.8312674Z remote: Counting objects: 63% (1107/1756) 2024-08-20T22:05:07.8313198Z remote: Counting objects: 64% (1124/1756) 2024-08-20T22:05:07.8313731Z remote: Counting objects: 65% (1142/1756) 2024-08-20T22:05:07.8314254Z remote: Counting objects: 66% (1159/1756) 2024-08-20T22:05:07.8314778Z remote: Counting objects: 67% (1177/1756) 2024-08-20T22:05:07.8315292Z remote: Counting objects: 68% (1195/1756) 2024-08-20T22:05:07.8315811Z remote: Counting objects: 69% (1212/1756) 2024-08-20T22:05:07.8316331Z remote: Counting objects: 70% (1230/1756) 2024-08-20T22:05:07.8316853Z remote: Counting objects: 71% (1247/1756) 2024-08-20T22:05:07.8317368Z remote: Counting objects: 72% (1265/1756) 2024-08-20T22:05:07.8317888Z remote: Counting objects: 73% (1282/1756) 2024-08-20T22:05:07.8318403Z remote: Counting objects: 74% (1300/1756) 2024-08-20T22:05:07.8318915Z remote: Counting objects: 75% (1317/1756) 2024-08-20T22:05:07.8319436Z remote: Counting objects: 76% (1335/1756) 2024-08-20T22:05:07.8320055Z remote: Counting objects: 77% (1353/1756) 2024-08-20T22:05:07.8320581Z remote: Counting objects: 78% (1370/1756) 2024-08-20T22:05:07.8321089Z remote: Counting objects: 79% (1388/1756) 2024-08-20T22:05:07.8321602Z remote: Counting objects: 80% (1405/1756) 2024-08-20T22:05:07.8322118Z remote: Counting objects: 81% (1423/1756) 2024-08-20T22:05:07.8322631Z remote: Counting objects: 82% (1440/1756) 2024-08-20T22:05:07.8323138Z remote: Counting objects: 83% (1458/1756) 2024-08-20T22:05:07.8323651Z remote: Counting objects: 84% (1476/1756) 2024-08-20T22:05:07.8324162Z remote: Counting objects: 85% (1493/1756) 2024-08-20T22:05:07.8324674Z remote: Counting objects: 86% (1511/1756) 2024-08-20T22:05:07.8325181Z remote: Counting objects: 87% (1528/1756) 2024-08-20T22:05:07.8325692Z remote: Counting objects: 88% (1546/1756) 2024-08-20T22:05:07.8326204Z remote: Counting objects: 89% (1563/1756) 2024-08-20T22:05:07.8326830Z remote: Counting objects: 90% (1581/1756) 2024-08-20T22:05:07.8327337Z remote: Counting objects: 91% (1598/1756) 2024-08-20T22:05:07.8327848Z remote: Counting objects: 92% (1616/1756) 2024-08-20T22:05:07.8328367Z remote: Counting objects: 93% (1634/1756) 2024-08-20T22:05:07.8328882Z remote: Counting objects: 94% (1651/1756) 2024-08-20T22:05:07.8329393Z remote: Counting objects: 95% (1669/1756) 2024-08-20T22:05:07.8329905Z remote: Counting objects: 96% (1686/1756) 2024-08-20T22:05:07.8330420Z remote: Counting objects: 97% (1704/1756) 2024-08-20T22:05:07.8330933Z remote: Counting objects: 98% (1721/1756) 2024-08-20T22:05:07.8331441Z remote: Counting objects: 99% (1739/1756) 2024-08-20T22:05:07.8331955Z remote: Counting objects: 100% (1756/1756) 2024-08-20T22:05:07.8332508Z remote: Counting objects: 100% (1756/1756), done. 2024-08-20T22:05:07.8428474Z remote: Compressing objects: 0% (1/837) 2024-08-20T22:05:07.8691302Z remote: Compressing objects: 1% (9/837) 2024-08-20T22:05:07.9029716Z remote: Compressing objects: 2% (17/837) 2024-08-20T22:05:07.9425366Z remote: Compressing objects: 3% (26/837) 2024-08-20T22:05:08.0124055Z remote: Compressing objects: 4% (34/837) 2024-08-20T22:05:08.0545483Z remote: Compressing objects: 5% (42/837) 2024-08-20T22:05:08.1107016Z remote: Compressing objects: 6% (51/837) 2024-08-20T22:05:08.1421249Z remote: Compressing objects: 7% (59/837) 2024-08-20T22:05:08.1783585Z remote: Compressing objects: 8% (67/837) 2024-08-20T22:05:08.1986325Z remote: Compressing objects: 9% (76/837) 2024-08-20T22:05:08.2237853Z remote: Compressing objects: 10% (84/837) 2024-08-20T22:05:08.2433696Z remote: Compressing objects: 11% (93/837) 2024-08-20T22:05:08.2602691Z remote: Compressing objects: 12% (101/837) 2024-08-20T22:05:08.2758523Z remote: Compressing objects: 13% (109/837) 2024-08-20T22:05:08.2872586Z remote: Compressing objects: 14% (118/837) 2024-08-20T22:05:08.2955728Z remote: Compressing objects: 15% (126/837) 2024-08-20T22:05:08.3041802Z remote: Compressing objects: 16% (134/837) 2024-08-20T22:05:08.3097798Z remote: Compressing objects: 17% (143/837) 2024-08-20T22:05:08.3146185Z remote: Compressing objects: 18% (151/837) 2024-08-20T22:05:08.3186685Z remote: Compressing objects: 19% (160/837) 2024-08-20T22:05:08.3216420Z remote: Compressing objects: 20% (168/837) 2024-08-20T22:05:08.3237487Z remote: Compressing objects: 21% (176/837) 2024-08-20T22:05:08.3240706Z remote: Compressing objects: 22% (185/837) 2024-08-20T22:05:08.3245655Z remote: Compressing objects: 23% (193/837) 2024-08-20T22:05:08.3250990Z remote: Compressing objects: 24% (201/837) 2024-08-20T22:05:08.3255514Z remote: Compressing objects: 25% (210/837) 2024-08-20T22:05:08.3263084Z remote: Compressing objects: 26% (218/837) 2024-08-20T22:05:08.3273410Z remote: Compressing objects: 27% (226/837) 2024-08-20T22:05:08.3292908Z remote: Compressing objects: 28% (235/837) 2024-08-20T22:05:08.3297015Z remote: Compressing objects: 29% (243/837) 2024-08-20T22:05:08.3308254Z remote: Compressing objects: 30% (252/837) 2024-08-20T22:05:08.3317401Z remote: Compressing objects: 31% (260/837) 2024-08-20T22:05:08.3324582Z remote: Compressing objects: 32% (268/837) 2024-08-20T22:05:08.3329753Z remote: Compressing objects: 33% (277/837) 2024-08-20T22:05:08.3334280Z remote: Compressing objects: 34% (285/837) 2024-08-20T22:05:08.3341691Z remote: Compressing objects: 35% (293/837) 2024-08-20T22:05:08.3348629Z remote: Compressing objects: 36% (302/837) 2024-08-20T22:05:08.3357048Z remote: Compressing objects: 37% (310/837) 2024-08-20T22:05:08.3362078Z remote: Compressing objects: 38% (319/837) 2024-08-20T22:05:08.3368235Z remote: Compressing objects: 39% (327/837) 2024-08-20T22:05:08.3373841Z remote: Compressing objects: 40% (335/837) 2024-08-20T22:05:08.3380350Z remote: Compressing objects: 41% (344/837) 2024-08-20T22:05:08.3387802Z remote: Compressing objects: 42% (352/837) 2024-08-20T22:05:08.3392657Z remote: Compressing objects: 43% (360/837) 2024-08-20T22:05:08.3399465Z remote: Compressing objects: 44% (369/837) 2024-08-20T22:05:08.3404844Z remote: Compressing objects: 45% (377/837) 2024-08-20T22:05:08.3408304Z remote: Compressing objects: 46% (386/837) 2024-08-20T22:05:08.3412046Z remote: Compressing objects: 47% (394/837) 2024-08-20T22:05:08.3414819Z remote: Compressing objects: 48% (402/837) 2024-08-20T22:05:08.3417311Z remote: Compressing objects: 49% (411/837) 2024-08-20T22:05:08.3419938Z remote: Compressing objects: 50% (419/837) 2024-08-20T22:05:08.3423803Z remote: Compressing objects: 51% (427/837) 2024-08-20T22:05:08.3426436Z remote: Compressing objects: 52% (436/837) 2024-08-20T22:05:08.3429188Z remote: Compressing objects: 53% (444/837) 2024-08-20T22:05:08.3429880Z remote: Compressing objects: 54% (452/837) 2024-08-20T22:05:08.3432078Z remote: Compressing objects: 55% (461/837) 2024-08-20T22:05:08.3432762Z remote: Compressing objects: 56% (469/837) 2024-08-20T22:05:08.3434681Z remote: Compressing objects: 57% (478/837) 2024-08-20T22:05:08.3435384Z remote: Compressing objects: 58% (486/837) 2024-08-20T22:05:08.3435983Z remote: Compressing objects: 59% (494/837) 2024-08-20T22:05:08.3436726Z remote: Compressing objects: 60% (503/837) 2024-08-20T22:05:08.3437421Z remote: Compressing objects: 61% (511/837) 2024-08-20T22:05:08.3438102Z remote: Compressing objects: 62% (519/837) 2024-08-20T22:05:08.3438631Z remote: Compressing objects: 63% (528/837) 2024-08-20T22:05:08.3441410Z remote: Compressing objects: 64% (536/837) 2024-08-20T22:05:08.3447063Z remote: Compressing objects: 65% (545/837) 2024-08-20T22:05:08.3452112Z remote: Compressing objects: 66% (553/837) 2024-08-20T22:05:08.3456520Z remote: Compressing objects: 67% (561/837) 2024-08-20T22:05:08.3460618Z remote: Compressing objects: 68% (570/837) 2024-08-20T22:05:08.3465393Z remote: Compressing objects: 69% (578/837) 2024-08-20T22:05:08.3469022Z remote: Compressing objects: 70% (586/837) 2024-08-20T22:05:08.3471626Z remote: Compressing objects: 71% (595/837) 2024-08-20T22:05:08.3474729Z remote: Compressing objects: 72% (603/837) 2024-08-20T22:05:08.3478255Z remote: Compressing objects: 73% (612/837) 2024-08-20T22:05:08.3481383Z remote: Compressing objects: 74% (620/837) 2024-08-20T22:05:08.3484310Z remote: Compressing objects: 75% (628/837) 2024-08-20T22:05:08.3487024Z remote: Compressing objects: 76% (637/837) 2024-08-20T22:05:08.3489746Z remote: Compressing objects: 77% (645/837) 2024-08-20T22:05:08.3492294Z remote: Compressing objects: 78% (653/837) 2024-08-20T22:05:08.3494751Z remote: Compressing objects: 79% (662/837) 2024-08-20T22:05:08.3497379Z remote: Compressing objects: 80% (670/837) 2024-08-20T22:05:08.3498082Z remote: Compressing objects: 81% (678/837) 2024-08-20T22:05:08.3499991Z remote: Compressing objects: 82% (687/837) 2024-08-20T22:05:08.3502810Z remote: Compressing objects: 83% (695/837) 2024-08-20T22:05:08.3505228Z remote: Compressing objects: 84% (704/837) 2024-08-20T22:05:08.3505910Z remote: Compressing objects: 85% (712/837) 2024-08-20T22:05:08.3507905Z remote: Compressing objects: 86% (720/837) 2024-08-20T22:05:08.3508604Z remote: Compressing objects: 87% (729/837) 2024-08-20T22:05:08.3510328Z remote: Compressing objects: 88% (737/837) 2024-08-20T22:05:08.3512860Z remote: Compressing objects: 89% (745/837) 2024-08-20T22:05:08.3515391Z remote: Compressing objects: 90% (754/837) 2024-08-20T22:05:08.3516084Z remote: Compressing objects: 91% (762/837) 2024-08-20T22:05:08.3517950Z remote: Compressing objects: 92% (771/837) 2024-08-20T22:05:08.3518638Z remote: Compressing objects: 93% (779/837) 2024-08-20T22:05:08.3520739Z remote: Compressing objects: 94% (787/837) 2024-08-20T22:05:08.3521427Z remote: Compressing objects: 95% (796/837) 2024-08-20T22:05:08.3521990Z remote: Compressing objects: 96% (804/837) 2024-08-20T22:05:08.3522957Z remote: Compressing objects: 97% (812/837) 2024-08-20T22:05:08.3523638Z remote: Compressing objects: 98% (821/837) 2024-08-20T22:05:08.3525787Z remote: Compressing objects: 99% (829/837) 2024-08-20T22:05:08.3526479Z remote: Compressing objects: 100% (837/837) 2024-08-20T22:05:08.3527106Z remote: Compressing objects: 100% (837/837), done. 2024-08-20T22:05:32.0016546Z remote: Total 1010208 (delta 1154), reused 1394 (delta 916), pack-reused 1008452 (from 1) 2024-08-20T22:05:54.7232517Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T22:05:54.7268778Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:05:54.7274519Z ##[endgroup] 2024-08-20T22:05:54.7275089Z ##[group]Determining the checkout info 2024-08-20T22:05:54.7276271Z ##[endgroup] 2024-08-20T22:05:54.7277186Z ##[group]Checking out the ref 2024-08-20T22:05:54.7279760Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:05:56.4557506Z ##[endgroup] 2024-08-20T22:05:56.4558144Z ##[group]Setting up auth for fetching submodules 2024-08-20T22:05:56.4562381Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:05:56.4626104Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T22:05:56.4667054Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T22:05:56.4707788Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T22:05:56.4746329Z ##[endgroup] 2024-08-20T22:05:56.4747020Z ##[group]Fetching submodules 2024-08-20T22:05:56.4750078Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T22:05:56.5129018Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T22:05:56.5495584Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T22:05:56.5497201Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T22:05:56.5500771Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T22:05:56.5504915Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T22:05:56.5508683Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T22:05:56.5512177Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T22:05:56.5516078Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T22:05:56.5520143Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T22:05:56.5524317Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T22:05:56.5528607Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T22:05:56.5532785Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T22:05:56.5537740Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T22:05:56.5542075Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T22:05:56.5546434Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T22:05:56.5551387Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T22:05:56.5555834Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:05:56.5564360Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T22:05:56.5568547Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T22:05:56.5572494Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T22:05:56.5577102Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T22:05:56.5582002Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T22:05:56.5587050Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T22:05:56.5591656Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T22:05:56.5596409Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T22:05:56.5601120Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T22:05:56.5606452Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T22:05:56.5611128Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T22:05:56.5616352Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T22:05:56.5621367Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T22:05:56.5626962Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T22:05:56.5632113Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T22:05:56.5637860Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T22:05:56.5643225Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T22:05:56.5652411Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T22:05:56.5687384Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T22:05:56.8498344Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T22:05:57.0217097Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T22:05:57.2040370Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T22:05:57.4306995Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T22:05:59.3684865Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T22:06:10.7219241Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T22:06:11.1365769Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T22:06:11.5801984Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T22:06:12.1831208Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T22:06:13.4004611Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T22:06:15.5231284Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T22:06:21.4762812Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T22:06:23.0032026Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T22:06:24.7629556Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T22:06:26.0865192Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T22:06:26.5124662Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T22:06:26.8477746Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T22:06:28.0126965Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T22:06:28.4005563Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T22:06:28.6679265Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T22:06:30.3307058Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T22:06:31.0361432Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T22:06:31.7261399Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T22:06:37.7965375Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T22:06:39.7414030Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T22:06:43.9704799Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T22:06:44.1962303Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T22:06:53.3138630Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T22:06:53.4747236Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T22:06:53.6600969Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T22:06:54.5681390Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T22:06:54.8618834Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T22:06:55.4795600Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T22:06:55.8848190Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T22:06:55.8998773Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T22:06:55.9118455Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T22:06:55.9437646Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T22:06:55.9890292Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T22:06:57.1704433Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T22:06:57.1985403Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T22:06:57.2508497Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T22:06:57.3620501Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T22:06:57.4027390Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T22:06:58.0124746Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T22:06:58.2955942Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T22:06:58.3901974Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T22:06:58.3924819Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:06:58.3928054Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:06:58.3932594Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:06:58.3935812Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:06:58.3940018Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:06:58.3972339Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T22:06:59.3255535Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T22:06:59.9500482Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T22:07:02.1261526Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T22:07:03.2283695Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T22:07:03.5820596Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T22:07:03.6932691Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T22:07:04.2067192Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T22:07:04.2777156Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T22:07:04.2929367Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T22:07:04.4466185Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T22:07:04.4928384Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T22:07:04.5393182Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T22:07:04.5712003Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T22:07:04.6250468Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:07:04.6412104Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T22:07:04.6432157Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T22:07:04.6464478Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T22:07:17.9870656Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T22:07:18.0093909Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T22:07:18.1095956Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T22:07:18.1119073Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:07:18.1122069Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:07:18.1125520Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:07:18.1158719Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T22:07:18.6598969Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T22:07:19.8601022Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T22:07:21.1071077Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T22:07:21.1091801Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:07:21.1095110Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:07:21.1098602Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:07:21.1102325Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:07:21.1107052Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:07:21.1110879Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:07:21.1114903Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:07:21.1119133Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:07:21.1152124Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T22:07:21.9871899Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T22:07:22.3402175Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T22:07:23.5112665Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T22:07:23.7844777Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T22:07:24.3616122Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T22:07:25.3939745Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T22:07:32.8433718Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T22:07:33.3170340Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T22:07:33.3407479Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T22:07:33.3845929Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T22:07:33.4016127Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T22:07:33.4035933Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:07:33.4069423Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T22:07:33.7300378Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T22:07:33.7527243Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T22:07:33.8016458Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T22:07:33.9233178Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T22:07:33.9434065Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T22:07:33.9891908Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T22:07:34.0555162Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T22:07:34.1017827Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T22:07:34.1313904Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T22:07:34.2580439Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T22:07:34.7752750Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T22:07:34.7790554Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T22:07:34.7793626Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T22:07:34.7828203Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T22:07:35.2572296Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T22:07:36.1964887Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T22:07:36.2381957Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T22:07:36.3309609Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T22:07:36.3333171Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:07:36.3336904Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:07:36.3340730Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:07:36.3344754Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:07:36.3348869Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:07:36.3352699Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:07:36.3356630Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:07:36.3360411Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:07:36.3392953Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T22:07:36.7645162Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T22:07:37.7990718Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T22:07:38.0875705Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T22:07:44.2109015Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T22:07:44.4654846Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T22:07:44.6617896Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T22:07:44.9458258Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T22:07:51.7368357Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T22:07:51.7843344Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T22:07:51.8033816Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T22:07:51.9262112Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T22:07:51.9425237Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T22:07:51.9611994Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T22:07:51.9815491Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T22:07:51.9835090Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:07:51.9838766Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:07:51.9869855Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T22:07:53.8761948Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T22:07:55.2169918Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T22:07:55.2718531Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:07:55.9523888Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T22:07:55.9671768Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T22:07:56.2786676Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T22:07:56.2808930Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:07:56.2812287Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T22:07:56.2845899Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T22:07:56.7356192Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T22:07:57.7749666Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T22:07:57.8574795Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T22:07:57.8693753Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T22:07:57.8854410Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T22:07:57.9309380Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T22:07:57.9656139Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T22:07:58.0148772Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T22:07:58.0488947Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T22:07:58.0508782Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:07:58.0512256Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:07:58.0515435Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:07:58.0519561Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:07:58.0551711Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T22:07:59.1314780Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T22:07:59.3715707Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T22:08:00.5525413Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T22:08:01.5347210Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T22:08:01.5545063Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T22:08:01.6337535Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T22:08:01.6685778Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T22:08:01.6705120Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:01.6737354Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T22:08:01.8830303Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T22:08:01.8878733Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T22:08:01.9248313Z Entering 'android/libs/fbjni' 2024-08-20T22:08:01.9302274Z Entering 'third_party/FP16' 2024-08-20T22:08:01.9355503Z Entering 'third_party/FXdiv' 2024-08-20T22:08:01.9407879Z Entering 'third_party/NNPACK' 2024-08-20T22:08:01.9462024Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:01.9515415Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:01.9584579Z Entering 'third_party/benchmark' 2024-08-20T22:08:01.9636849Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:01.9688839Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:01.9742078Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:01.9793959Z Entering 'third_party/cutlass' 2024-08-20T22:08:01.9854874Z Entering 'third_party/eigen' 2024-08-20T22:08:01.9908840Z Entering 'third_party/fbgemm' 2024-08-20T22:08:01.9962015Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:02.0013003Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:02.0064367Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:02.0121974Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:02.0172283Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:02.0224813Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:02.0280280Z Entering 'third_party/fmt' 2024-08-20T22:08:02.0332144Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:02.0389820Z Entering 'third_party/gloo' 2024-08-20T22:08:02.0441533Z Entering 'third_party/googletest' 2024-08-20T22:08:02.0494277Z Entering 'third_party/ideep' 2024-08-20T22:08:02.0545366Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:02.0604954Z Entering 'third_party/ittapi' 2024-08-20T22:08:02.0657083Z Entering 'third_party/kineto' 2024-08-20T22:08:02.0708122Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:02.0759373Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:02.0811590Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:02.0863747Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:02.0915308Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:02.0966328Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:02.1021750Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:02.1076435Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:02.1128207Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:02.1181129Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:02.1235552Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:02.1286170Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:02.1339866Z Entering 'third_party/mimalloc' 2024-08-20T22:08:02.1392936Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:02.1445931Z Entering 'third_party/nlohmann' 2024-08-20T22:08:02.1498630Z Entering 'third_party/onnx' 2024-08-20T22:08:02.1565397Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:02.1616916Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:02.1674150Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:02.1727099Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:02.1777445Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:02.1831617Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:02.1881763Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:02.1933658Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:02.1988777Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:02.2040049Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:02.2089164Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:02.2144764Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:02.2197807Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:02.2271099Z Entering 'third_party/pocketfft' 2024-08-20T22:08:02.2323342Z Entering 'third_party/protobuf' 2024-08-20T22:08:02.2382127Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:02.2432618Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:02.2491619Z Entering 'third_party/psimd' 2024-08-20T22:08:02.2545848Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:02.2598159Z Entering 'third_party/pybind11' 2024-08-20T22:08:02.2651908Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:02.2704903Z Entering 'third_party/sleef' 2024-08-20T22:08:02.2757832Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:02.2809787Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:02.2863355Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:02.2914431Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:02.2966646Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:02.3015900Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:02.3093378Z ##[endgroup] 2024-08-20T22:08:02.3096843Z ##[group]Persisting credentials for submodules 2024-08-20T22:08:02.3099500Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T22:08:02.3471525Z Entering 'android/libs/fbjni' 2024-08-20T22:08:02.3540842Z Entering 'third_party/FP16' 2024-08-20T22:08:02.3609340Z Entering 'third_party/FXdiv' 2024-08-20T22:08:02.3677341Z Entering 'third_party/NNPACK' 2024-08-20T22:08:02.3745191Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:02.3813111Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:02.3901888Z Entering 'third_party/benchmark' 2024-08-20T22:08:02.3973356Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:02.4041841Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:02.4110640Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:02.4178908Z Entering 'third_party/cutlass' 2024-08-20T22:08:02.4254711Z Entering 'third_party/eigen' 2024-08-20T22:08:02.4324619Z Entering 'third_party/fbgemm' 2024-08-20T22:08:02.4392642Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:02.4460242Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:02.4528245Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:02.4601274Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:02.4668765Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:02.4737730Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:02.4814056Z Entering 'third_party/fmt' 2024-08-20T22:08:02.4882808Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:02.4950770Z Entering 'third_party/gloo' 2024-08-20T22:08:02.5019023Z Entering 'third_party/googletest' 2024-08-20T22:08:02.5087287Z Entering 'third_party/ideep' 2024-08-20T22:08:02.5158168Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:02.5232782Z Entering 'third_party/ittapi' 2024-08-20T22:08:02.5303432Z Entering 'third_party/kineto' 2024-08-20T22:08:02.5370636Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:02.5435756Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:02.5504669Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:02.5572208Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:02.5641197Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:02.5708393Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:02.5780309Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:02.5848855Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:02.5915919Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:02.5989555Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:02.6060124Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:02.6128182Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:02.6198330Z Entering 'third_party/mimalloc' 2024-08-20T22:08:02.6266789Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:02.6334385Z Entering 'third_party/nlohmann' 2024-08-20T22:08:02.6408561Z Entering 'third_party/onnx' 2024-08-20T22:08:02.6490651Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:02.6559411Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:02.6633449Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:02.6703576Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:02.6771162Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:02.6837979Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:02.6903417Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:02.6971718Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:02.7038536Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:02.7104734Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:02.7170145Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:02.7237632Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:02.7307686Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:02.7399156Z Entering 'third_party/pocketfft' 2024-08-20T22:08:02.7466990Z Entering 'third_party/protobuf' 2024-08-20T22:08:02.7536028Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:02.7604011Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:02.7675134Z Entering 'third_party/psimd' 2024-08-20T22:08:02.7746612Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:02.7818219Z Entering 'third_party/pybind11' 2024-08-20T22:08:02.7886235Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:02.7954561Z Entering 'third_party/sleef' 2024-08-20T22:08:02.8022281Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:02.8089948Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:02.8157253Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:02.8222057Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:02.8289769Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:02.8357078Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:02.8451587Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T22:08:02.8818167Z Entering 'android/libs/fbjni' 2024-08-20T22:08:02.8884724Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T22:08:02.8906157Z Entering 'third_party/FP16' 2024-08-20T22:08:02.8971145Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T22:08:02.8991652Z Entering 'third_party/FXdiv' 2024-08-20T22:08:02.9055979Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T22:08:02.9078305Z Entering 'third_party/NNPACK' 2024-08-20T22:08:02.9140572Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T22:08:02.9163163Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:02.9225601Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T22:08:02.9249865Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:02.9311061Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T22:08:02.9348903Z Entering 'third_party/benchmark' 2024-08-20T22:08:02.9410418Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:02.9431724Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:02.9493349Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T22:08:02.9514611Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:02.9576453Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:08:02.9598626Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:02.9660923Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T22:08:02.9682351Z Entering 'third_party/cutlass' 2024-08-20T22:08:02.9744903Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:08:02.9774401Z Entering 'third_party/eigen' 2024-08-20T22:08:02.9836122Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T22:08:02.9859869Z Entering 'third_party/fbgemm' 2024-08-20T22:08:02.9922131Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T22:08:02.9943228Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:03.0006309Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T22:08:03.0027850Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:03.0090185Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:08:03.0109414Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:03.0170918Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:08:03.0198562Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:03.0260261Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.0280836Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:03.0342671Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T22:08:03.0366094Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:03.0427707Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T22:08:03.0451805Z Entering 'third_party/fmt' 2024-08-20T22:08:03.0512614Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:08:03.0533737Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:03.0596336Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T22:08:03.0617985Z Entering 'third_party/gloo' 2024-08-20T22:08:03.0680159Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T22:08:03.0702139Z Entering 'third_party/googletest' 2024-08-20T22:08:03.0763297Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.0784512Z Entering 'third_party/ideep' 2024-08-20T22:08:03.0847292Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T22:08:03.0866185Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:03.0927078Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T22:08:03.0956970Z Entering 'third_party/ittapi' 2024-08-20T22:08:03.1019696Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T22:08:03.1041632Z Entering 'third_party/kineto' 2024-08-20T22:08:03.1104109Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T22:08:03.1123864Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:03.1187612Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T22:08:03.1206996Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:03.1275094Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T22:08:03.1296577Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:03.1364938Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T22:08:03.1385229Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:03.1447709Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:08:03.1468498Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:03.1535589Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T22:08:03.1556284Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:03.1619729Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T22:08:03.1643996Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:03.1707886Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T22:08:03.1728231Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:03.1791348Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.1811889Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:03.1874960Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T22:08:03.1896633Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:03.1965426Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T22:08:03.1988782Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:03.2051442Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T22:08:03.2072317Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:03.2133438Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.2158160Z Entering 'third_party/mimalloc' 2024-08-20T22:08:03.2219741Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T22:08:03.2241898Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:03.2303508Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T22:08:03.2325077Z Entering 'third_party/nlohmann' 2024-08-20T22:08:03.2393008Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T22:08:03.2415249Z Entering 'third_party/onnx' 2024-08-20T22:08:03.2482468Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T22:08:03.2517075Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:03.2580687Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:03.2601810Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:03.2664992Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:08:03.2691192Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:03.2754668Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T22:08:03.2775890Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:03.2836996Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:03.2857474Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:03.2917760Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.2938607Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:03.3001899Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T22:08:03.3021515Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:03.3083970Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T22:08:03.3105284Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:03.3166594Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T22:08:03.3186532Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:03.3247921Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T22:08:03.3268201Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:03.3329347Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T22:08:03.3348538Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:03.3410083Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T22:08:03.3432376Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:03.3494580Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T22:08:03.3517752Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:03.3580622Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T22:08:03.3621813Z Entering 'third_party/pocketfft' 2024-08-20T22:08:03.3683796Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T22:08:03.3704798Z Entering 'third_party/protobuf' 2024-08-20T22:08:03.3766525Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T22:08:03.3791128Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:03.3853359Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:03.3874048Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:03.3935695Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.3961358Z Entering 'third_party/psimd' 2024-08-20T22:08:03.4023426Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T22:08:03.4046014Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:03.4110119Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T22:08:03.4131826Z Entering 'third_party/pybind11' 2024-08-20T22:08:03.4194199Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:08:03.4215947Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:03.4277081Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T22:08:03.4298285Z Entering 'third_party/sleef' 2024-08-20T22:08:03.4359851Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T22:08:03.4381408Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:03.4442946Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T22:08:03.4462946Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:03.4523580Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:03.4544407Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:03.4604376Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T22:08:03.4624177Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:03.4685462Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T22:08:03.4705771Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:03.4767313Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:08:03.4786136Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:03.4854176Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T22:08:03.5578153Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T22:08:03.5945404Z Entering 'android/libs/fbjni' 2024-08-20T22:08:03.5997038Z Entering 'third_party/FP16' 2024-08-20T22:08:03.6050157Z Entering 'third_party/FXdiv' 2024-08-20T22:08:03.6103279Z Entering 'third_party/NNPACK' 2024-08-20T22:08:03.6156941Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:03.6208585Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:03.6277226Z Entering 'third_party/benchmark' 2024-08-20T22:08:03.6329040Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:03.6382442Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:03.6437717Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:03.6495213Z Entering 'third_party/cutlass' 2024-08-20T22:08:03.6556928Z Entering 'third_party/eigen' 2024-08-20T22:08:03.6611601Z Entering 'third_party/fbgemm' 2024-08-20T22:08:03.6663388Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:03.6714739Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:03.6766457Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:03.6823384Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:03.6874439Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:03.6926888Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:03.6983384Z Entering 'third_party/fmt' 2024-08-20T22:08:03.7034282Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:03.7090830Z Entering 'third_party/gloo' 2024-08-20T22:08:03.7144379Z Entering 'third_party/googletest' 2024-08-20T22:08:03.7198294Z Entering 'third_party/ideep' 2024-08-20T22:08:03.7250481Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:03.7307784Z Entering 'third_party/ittapi' 2024-08-20T22:08:03.7361694Z Entering 'third_party/kineto' 2024-08-20T22:08:03.7413858Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:03.7464220Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:03.7517407Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:03.7569551Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:03.7619478Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:03.7669141Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:03.7723476Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:03.7774531Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:03.7824650Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:03.7876385Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:03.7929916Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:03.7981522Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:03.8033904Z Entering 'third_party/mimalloc' 2024-08-20T22:08:03.8091001Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:03.8143718Z Entering 'third_party/nlohmann' 2024-08-20T22:08:03.8196503Z Entering 'third_party/onnx' 2024-08-20T22:08:03.8267144Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:03.8319221Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:03.8376985Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:03.8431191Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:03.8482284Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:03.8532163Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:03.8581552Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:03.8633148Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:03.8682477Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:03.8732684Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:03.8782246Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:03.8835188Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:03.8890243Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:03.8961614Z Entering 'third_party/pocketfft' 2024-08-20T22:08:03.9015798Z Entering 'third_party/protobuf' 2024-08-20T22:08:03.9071874Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:03.9121775Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:03.9177317Z Entering 'third_party/psimd' 2024-08-20T22:08:03.9230933Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:03.9284956Z Entering 'third_party/pybind11' 2024-08-20T22:08:03.9337813Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:03.9390653Z Entering 'third_party/sleef' 2024-08-20T22:08:03.9443772Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:03.9495479Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:03.9546209Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:03.9601042Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:03.9652055Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:03.9700319Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:03.9778938Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T22:08:04.0148894Z Entering 'android/libs/fbjni' 2024-08-20T22:08:04.0200680Z Entering 'third_party/FP16' 2024-08-20T22:08:04.0254110Z Entering 'third_party/FXdiv' 2024-08-20T22:08:04.0307657Z Entering 'third_party/NNPACK' 2024-08-20T22:08:04.0361576Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:04.0414629Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:04.0486328Z Entering 'third_party/benchmark' 2024-08-20T22:08:04.0539588Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:04.0592610Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:04.0646501Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:04.0700036Z Entering 'third_party/cutlass' 2024-08-20T22:08:04.0762539Z Entering 'third_party/eigen' 2024-08-20T22:08:04.0818199Z Entering 'third_party/fbgemm' 2024-08-20T22:08:04.0871423Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:04.0922789Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:04.0975727Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:04.1033464Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:04.1086146Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:04.1145414Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:04.1202308Z Entering 'third_party/fmt' 2024-08-20T22:08:04.1255913Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:04.1311864Z Entering 'third_party/gloo' 2024-08-20T22:08:04.1366059Z Entering 'third_party/googletest' 2024-08-20T22:08:04.1419128Z Entering 'third_party/ideep' 2024-08-20T22:08:04.1471180Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:04.1530342Z Entering 'third_party/ittapi' 2024-08-20T22:08:04.1583371Z Entering 'third_party/kineto' 2024-08-20T22:08:04.1634258Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:04.1691389Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:04.1742317Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:04.1792444Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:04.1843043Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:04.1891706Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:04.1947097Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:04.1997773Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:04.2049926Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:04.2101841Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:04.2154211Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:04.2204753Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:04.2258497Z Entering 'third_party/mimalloc' 2024-08-20T22:08:04.2314433Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:04.2367391Z Entering 'third_party/nlohmann' 2024-08-20T22:08:04.2421120Z Entering 'third_party/onnx' 2024-08-20T22:08:04.2490221Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:04.2542548Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:04.2600447Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:04.2654987Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:04.2710466Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:04.2761478Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:04.2812920Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:04.2872391Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:04.2925812Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:04.2979975Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:04.3029171Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:04.3083233Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:04.3137219Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:04.3211298Z Entering 'third_party/pocketfft' 2024-08-20T22:08:04.3264936Z Entering 'third_party/protobuf' 2024-08-20T22:08:04.3319675Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:04.3371124Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:04.3426314Z Entering 'third_party/psimd' 2024-08-20T22:08:04.3480993Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:04.3535270Z Entering 'third_party/pybind11' 2024-08-20T22:08:04.3590164Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:04.3645269Z Entering 'third_party/sleef' 2024-08-20T22:08:04.3698974Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:04.3751878Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:04.3802977Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:04.3852946Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:04.3904452Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:04.3954279Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:04.4029048Z ##[endgroup] 2024-08-20T22:08:04.4080289Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T22:08:04.4119703Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T22:08:04.4333973Z Prepare all required actions 2024-08-20T22:08:04.4334441Z Getting action download info 2024-08-20T22:08:04.5674271Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T22:08:04.5674704Z env: 2024-08-20T22:08:04.5674996Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.5675346Z ##[endgroup] 2024-08-20T22:08:04.5764555Z ##[group]Run set -euo pipefail 2024-08-20T22:08:04.5765004Z set -euo pipefail 2024-08-20T22:08:04.5765410Z function get_ec2_metadata() { 2024-08-20T22:08:04.5765960Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T22:08:04.5766841Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T22:08:04.5767601Z  category=$1 2024-08-20T22:08:04.5768133Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T22:08:04.5768767Z  runner_name_str=i-00e21703921bea6d0 2024-08-20T22:08:04.5769264Z  if [[ -f /.inarc ]]; then 2024-08-20T22:08:04.5769765Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T22:08:04.5770335Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T22:08:04.5771033Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T22:08:04.5771631Z  else 2024-08-20T22:08:04.5772118Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T22:08:04.5772687Z  fi 2024-08-20T22:08:04.5772976Z } 2024-08-20T22:08:04.5773332Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T22:08:04.5773919Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T22:08:04.5774574Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T22:08:04.5775137Z echo "system info $(uname -a)" 2024-08-20T22:08:04.5786067Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:04.5786564Z env: 2024-08-20T22:08:04.5786854Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.5787242Z ##[endgroup] 2024-08-20T22:08:04.5885679Z ami-id: ami-06c68f701d8090592 2024-08-20T22:08:04.5946665Z instance-id: i-00e21703921bea6d0 2024-08-20T22:08:04.6007021Z instance-type: g5.4xlarge 2024-08-20T22:08:04.6020728Z system info Linux ip-10-0-46-207.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T22:08:04.6044882Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:08:04.6045784Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:08:04.6055001Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:04.6055498Z env: 2024-08-20T22:08:04.6055783Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.6056136Z ##[endgroup] 2024-08-20T22:08:04.6137013Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T22:08:04.6137630Z if systemctl is-active --quiet docker; then 2024-08-20T22:08:04.6138171Z  echo "Docker daemon is running..."; 2024-08-20T22:08:04.6138821Z else 2024-08-20T22:08:04.6139317Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T22:08:04.6139905Z fi 2024-08-20T22:08:04.6149250Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:04.6149889Z env: 2024-08-20T22:08:04.6150265Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.6150665Z ##[endgroup] 2024-08-20T22:08:04.6243897Z Docker daemon is running... 2024-08-20T22:08:04.6293841Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:08:04.6294416Z with: 2024-08-20T22:08:04.6294704Z shell: bash 2024-08-20T22:08:04.6295015Z timeout_minutes: 5 2024-08-20T22:08:04.6295359Z max_attempts: 3 2024-08-20T22:08:04.6295695Z retry_wait_seconds: 30 2024-08-20T22:08:04.6298774Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T22:08:04.6301733Z polling_interval_seconds: 1 2024-08-20T22:08:04.6302121Z warning_on_retry: true 2024-08-20T22:08:04.6302607Z continue_on_error: false 2024-08-20T22:08:04.6302960Z env: 2024-08-20T22:08:04.6303245Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:04.6303612Z AWS_RETRY_MODE: standard 2024-08-20T22:08:04.6303965Z AWS_MAX_ATTEMPTS: 5 2024-08-20T22:08:04.6304313Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T22:08:04.6304700Z ##[endgroup] 2024-08-20T22:08:05.9892191Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:08:05.9893055Z Configure a credential helper to remove this warning. See 2024-08-20T22:08:05.9893914Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:08:05.9894422Z 2024-08-20T22:08:05.9894545Z Login Succeeded 2024-08-20T22:08:06.6865644Z Command completed after 1 attempt(s). 2024-08-20T22:08:06.6927952Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:08:06.6928679Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:08:06.6929337Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:08:06.6938873Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:06.6939386Z env: 2024-08-20T22:08:06.6939668Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:06.6940020Z ##[endgroup] 2024-08-20T22:08:06.7035606Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:08:06.7036363Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:08:06.7037465Z # shellcheck disable=SC2046 2024-08-20T22:08:06.7038014Z docker stop $(docker ps -q) || true 2024-08-20T22:08:06.7038549Z # Prune all of the docker images 2024-08-20T22:08:06.7038995Z docker system prune -af 2024-08-20T22:08:06.7047945Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:06.7048449Z env: 2024-08-20T22:08:06.7048734Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:06.7049087Z ##[endgroup] 2024-08-20T22:08:06.7359631Z "docker stop" requires at least 1 argument. 2024-08-20T22:08:06.7360515Z See 'docker stop --help'. 2024-08-20T22:08:06.7360839Z 2024-08-20T22:08:06.7361061Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T22:08:06.7361427Z 2024-08-20T22:08:06.7361582Z Stop one or more running containers 2024-08-20T22:08:06.7539713Z Total reclaimed space: 0B 2024-08-20T22:08:06.7588648Z ##[group]Run set +e 2024-08-20T22:08:06.7589026Z set +e 2024-08-20T22:08:06.7589504Z set -x 2024-08-20T22:08:06.7589805Z  2024-08-20T22:08:06.7590129Z PT_DOMAIN=download.pytorch.org 2024-08-20T22:08:06.7590917Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T22:08:06.7591997Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T22:08:06.7592749Z # one is returned at random 2024-08-20T22:08:06.7593312Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T22:08:06.7593834Z  2024-08-20T22:08:06.7594175Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:08:06.7594807Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T22:08:06.7595758Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T22:08:06.7596318Z  2024-08-20T22:08:06.7596650Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:08:06.7597214Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T22:08:06.7597729Z  exit 1 2024-08-20T22:08:06.7598096Z  fi 2024-08-20T22:08:06.7598389Z fi 2024-08-20T22:08:06.7598676Z  2024-08-20T22:08:06.7599043Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T22:08:06.7599559Z  # Clean up any old records first 2024-08-20T22:08:06.7600074Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T22:08:06.7600535Z fi 2024-08-20T22:08:06.7600815Z  2024-08-20T22:08:06.7601255Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T22:08:06.7601809Z cat /etc/hosts 2024-08-20T22:08:06.7610873Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:06.7611366Z env: 2024-08-20T22:08:06.7611662Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:06.7612021Z ##[endgroup] 2024-08-20T22:08:06.7641520Z + PT_DOMAIN=download.pytorch.org 2024-08-20T22:08:06.7648374Z ++ dig -4 +short download.pytorch.org 2024-08-20T22:08:06.7648839Z ++ tail -n1 2024-08-20T22:08:06.8080969Z + RESOLVED_IP=18.160.10.22 2024-08-20T22:08:06.8081429Z + '[' -z 18.160.10.22 ']' 2024-08-20T22:08:06.8081885Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T22:08:06.8094927Z 108.138.64.116 download.pytorch.org 2024-08-20T22:08:06.8096888Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-20T22:08:06.9374072Z + echo '18.160.10.22 download.pytorch.org' 2024-08-20T22:08:06.9374593Z + sudo tee -a /etc/hosts 2024-08-20T22:08:06.9693709Z 18.160.10.22 download.pytorch.org 2024-08-20T22:08:06.9715507Z + cat /etc/hosts 2024-08-20T22:08:06.9725523Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T22:08:06.9739059Z ::1 localhost6 localhost6.localdomain6 2024-08-20T22:08:06.9739638Z 18.160.10.22 download.pytorch.org 2024-08-20T22:08:06.9890990Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T22:08:06.9891598Z with: 2024-08-20T22:08:06.9892680Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:06.9893891Z docker-build-dir: .ci/docker 2024-08-20T22:08:06.9894268Z working-directory: . 2024-08-20T22:08:06.9894730Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:06.9895248Z force-push: false 2024-08-20T22:08:06.9895551Z env: 2024-08-20T22:08:06.9895823Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:06.9896160Z ##[endgroup] 2024-08-20T22:08:06.9915318Z ##[group]Run set -ex 2024-08-20T22:08:06.9915689Z set -ex 2024-08-20T22:08:06.9915984Z  2024-08-20T22:08:06.9916538Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T22:08:06.9917501Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T22:08:06.9918463Z # job could then download the pre-built image as usual 2024-08-20T22:08:06.9919210Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T22:08:06.9919890Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:06.9920534Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:06.9921101Z  2024-08-20T22:08:06.9921617Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T22:08:06.9922250Z  exit 0 2024-08-20T22:08:06.9922568Z else 2024-08-20T22:08:06.9922948Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:06.9923416Z fi 2024-08-20T22:08:06.9923710Z  2024-08-20T22:08:06.9924198Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T22:08:06.9925041Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T22:08:06.9925824Z  # use it as it is, but first let's extract the tag 2024-08-20T22:08:06.9926527Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T22:08:06.9927255Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:06.9927956Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:06.9928515Z else 2024-08-20T22:08:06.9928960Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T22:08:06.9929630Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:06.9930516Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:06.9931262Z fi 2024-08-20T22:08:06.9940438Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:06.9940940Z env: 2024-08-20T22:08:06.9941221Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:06.9941571Z REPO_NAME: pytorch 2024-08-20T22:08:06.9942807Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:06.9944006Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:08:06.9944508Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:06.9945014Z ##[endgroup] 2024-08-20T22:08:06.9978292Z + [[ ! -d .ci/docker ]] 2024-08-20T22:08:06.9978825Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T22:08:06.9979202Z + echo skip=false 2024-08-20T22:08:06.9981211Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T22:08:06.9987402Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:06.9988805Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T22:08:07.0013987Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.0014716Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.0016161Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.0052112Z ##[group]Run set +e 2024-08-20T22:08:07.0052676Z set +e 2024-08-20T22:08:07.0053071Z set -x 2024-08-20T22:08:07.0053477Z  2024-08-20T22:08:07.0053933Z login() { 2024-08-20T22:08:07.0054663Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:08:07.0055584Z } 2024-08-20T22:08:07.0055951Z  2024-08-20T22:08:07.0056579Z retry () { 2024-08-20T22:08:07.0057158Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:08:07.0057711Z } 2024-08-20T22:08:07.0058050Z  2024-08-20T22:08:07.0058538Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:08:07.0059049Z  2024-08-20T22:08:07.0059616Z # Check if image already exists, if it does then skip building it 2024-08-20T22:08:07.0060458Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T22:08:07.0061063Z  exit 0 2024-08-20T22:08:07.0061469Z fi 2024-08-20T22:08:07.0061948Z  2024-08-20T22:08:07.0062623Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T22:08:07.0063643Z # be empty. The default action would be to continue rebuild the image 2024-08-20T22:08:07.0064532Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T22:08:07.0065295Z  # if we're on the base branch then use the parent commit 2024-08-20T22:08:07.0065962Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T22:08:07.0066539Z else 2024-08-20T22:08:07.0067102Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T22:08:07.0067907Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T22:08:07.0068518Z fi 2024-08-20T22:08:07.0068891Z  2024-08-20T22:08:07.0069325Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T22:08:07.0069965Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:07.0070575Z  2024-08-20T22:08:07.0071304Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T22:08:07.0072208Z  exit 0 2024-08-20T22:08:07.0072639Z fi 2024-08-20T22:08:07.0072982Z  2024-08-20T22:08:07.0073574Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T22:08:07.0074660Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T22:08:07.0075607Z  exit 1 2024-08-20T22:08:07.0075973Z fi 2024-08-20T22:08:07.0076384Z  2024-08-20T22:08:07.0077027Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T22:08:07.0078172Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T22:08:07.0079103Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T22:08:07.0080219Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T22:08:07.0081420Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T22:08:07.0082329Z fi 2024-08-20T22:08:07.0082756Z  2024-08-20T22:08:07.0083220Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:07.0092295Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:07.0092894Z env: 2024-08-20T22:08:07.0093229Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:07.0093852Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:08:07.0094419Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:08:07.0095821Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.0097236Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.0097901Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:07.0098534Z ##[endgroup] 2024-08-20T22:08:07.0128739Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:07.0129655Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:07.0131190Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:08:07.0133376Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:07.5492387Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:08:07.5494016Z Configure a credential helper to remove this warning. See 2024-08-20T22:08:07.5495161Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:08:07.5495915Z 2024-08-20T22:08:07.5496116Z Login Succeeded 2024-08-20T22:08:07.5519742Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.7774358Z { 2024-08-20T22:08:07.7775769Z "schemaVersion": 2, 2024-08-20T22:08:07.7776588Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T22:08:07.7777403Z "config": { 2024-08-20T22:08:07.7778205Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T22:08:07.7779134Z "size": 48711, 2024-08-20T22:08:07.7779870Z "digest": "sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb" 2024-08-20T22:08:07.7780591Z }, 2024-08-20T22:08:07.7781321Z "layers": [ 2024-08-20T22:08:07.7782613Z { 2024-08-20T22:08:07.7784498Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7785790Z "size": 28580681, 2024-08-20T22:08:07.7787103Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-20T22:08:07.7788117Z }, 2024-08-20T22:08:07.7788517Z { 2024-08-20T22:08:07.7789135Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7789799Z "size": 7943451, 2024-08-20T22:08:07.7790471Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-08-20T22:08:07.7791339Z }, 2024-08-20T22:08:07.7791643Z { 2024-08-20T22:08:07.7792161Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7792894Z "size": 55728572, 2024-08-20T22:08:07.7793524Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-08-20T22:08:07.7794181Z }, 2024-08-20T22:08:07.7794596Z { 2024-08-20T22:08:07.7795176Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7795935Z "size": 186, 2024-08-20T22:08:07.7796576Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-08-20T22:08:07.7797283Z }, 2024-08-20T22:08:07.7797655Z { 2024-08-20T22:08:07.7798293Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7798887Z "size": 6886, 2024-08-20T22:08:07.7799592Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-08-20T22:08:07.7800336Z }, 2024-08-20T22:08:07.7800995Z { 2024-08-20T22:08:07.7801529Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7802229Z "size": 1329001756, 2024-08-20T22:08:07.7802998Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-08-20T22:08:07.7803685Z }, 2024-08-20T22:08:07.7804082Z { 2024-08-20T22:08:07.7804631Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7805266Z "size": 62501, 2024-08-20T22:08:07.7805911Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-08-20T22:08:07.7806981Z }, 2024-08-20T22:08:07.7807334Z { 2024-08-20T22:08:07.7807822Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7808535Z "size": 1684, 2024-08-20T22:08:07.7809163Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-08-20T22:08:07.7809954Z }, 2024-08-20T22:08:07.7810309Z { 2024-08-20T22:08:07.7810839Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7811673Z "size": 1523, 2024-08-20T22:08:07.7812386Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-08-20T22:08:07.7813089Z }, 2024-08-20T22:08:07.7813435Z { 2024-08-20T22:08:07.7814019Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7814696Z "size": 2528295403, 2024-08-20T22:08:07.7815310Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-08-20T22:08:07.7816072Z }, 2024-08-20T22:08:07.7816509Z { 2024-08-20T22:08:07.7816994Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7817694Z "size": 86016, 2024-08-20T22:08:07.7818356Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-08-20T22:08:07.7819079Z }, 2024-08-20T22:08:07.7819480Z { 2024-08-20T22:08:07.7820020Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7820661Z "size": 1822, 2024-08-20T22:08:07.7821384Z "digest": "sha256:ac23c5fcc1ef310e9cd0d138b180281188e20a3e28c51cd14751622e3c7d92ea" 2024-08-20T22:08:07.7822046Z }, 2024-08-20T22:08:07.7822505Z { 2024-08-20T22:08:07.7823099Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7823814Z "size": 246761563, 2024-08-20T22:08:07.7824433Z "digest": "sha256:ca426ffc9d2bd051ad1ff5c4f02fd5d9be161e70953cc37882c815080c708f4d" 2024-08-20T22:08:07.7825219Z }, 2024-08-20T22:08:07.7825559Z { 2024-08-20T22:08:07.7826055Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7826741Z "size": 544, 2024-08-20T22:08:07.7827355Z "digest": "sha256:4373aa743caf6660c671295b837048c8350a2165f2336dc3b7b6c96462579e50" 2024-08-20T22:08:07.7828170Z }, 2024-08-20T22:08:07.7828477Z { 2024-08-20T22:08:07.7829001Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7829723Z "size": 1258, 2024-08-20T22:08:07.7830427Z "digest": "sha256:75b2abb16c685854c017d0cf75c626526c956734c656d9e78fdd4520317e9b3c" 2024-08-20T22:08:07.7831090Z }, 2024-08-20T22:08:07.7831502Z { 2024-08-20T22:08:07.7832012Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7832624Z "size": 484, 2024-08-20T22:08:07.7833284Z "digest": "sha256:ef1e3ef1e85f0124270b8506721219150e230184181f8207d64463bf5c67e656" 2024-08-20T22:08:07.7833966Z }, 2024-08-20T22:08:07.7834327Z { 2024-08-20T22:08:07.7834861Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7835503Z "size": 91711766, 2024-08-20T22:08:07.7836160Z "digest": "sha256:0bf61114d37b2d09065aa6852b1e78a32580a1ff011dd5d23f03b0f5c4e710af" 2024-08-20T22:08:07.7837174Z }, 2024-08-20T22:08:07.7837559Z { 2024-08-20T22:08:07.7838154Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7838845Z "size": 3390, 2024-08-20T22:08:07.7839638Z "digest": "sha256:2df849f72ba4258df10c9c45ace21a66d93b73196e78e96f12c13098822f60e2" 2024-08-20T22:08:07.7840322Z }, 2024-08-20T22:08:07.7840730Z { 2024-08-20T22:08:07.7841271Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7841869Z "size": 1909, 2024-08-20T22:08:07.7842565Z "digest": "sha256:34c47d63c5de27418739aa048b4dd781c3006d65691e9b903c95304d0509d89c" 2024-08-20T22:08:07.7843259Z }, 2024-08-20T22:08:07.7843602Z { 2024-08-20T22:08:07.7844243Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7844881Z "size": 701, 2024-08-20T22:08:07.7845509Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:07.7846287Z }, 2024-08-20T22:08:07.7846591Z { 2024-08-20T22:08:07.7847110Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7847824Z "size": 2864254879, 2024-08-20T22:08:07.7848449Z "digest": "sha256:6efb96fc87ea1bca54876d85331e5585e28f325a1d60ee1aef7811eabb74f933" 2024-08-20T22:08:07.7849279Z }, 2024-08-20T22:08:07.7849664Z { 2024-08-20T22:08:07.7850180Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7850796Z "size": 380, 2024-08-20T22:08:07.7851540Z "digest": "sha256:b119c4fc6c58d9e72744201ff4850b34184a94eb3566031b7d36e87ecce950a6" 2024-08-20T22:08:07.7852248Z }, 2024-08-20T22:08:07.7852637Z { 2024-08-20T22:08:07.7853123Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7853781Z "size": 12900, 2024-08-20T22:08:07.7854493Z "digest": "sha256:33b639226eeefa41d7bb26a8ecc20d271ceef45571dabfd80aaee3634926381b" 2024-08-20T22:08:07.7855208Z }, 2024-08-20T22:08:07.7855535Z { 2024-08-20T22:08:07.7856111Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7856753Z "size": 803, 2024-08-20T22:08:07.7857358Z "digest": "sha256:d3547bd43146930aa830d54562c98aa678e355cf5f84527e9717eb72fffe8ca3" 2024-08-20T22:08:07.7858232Z }, 2024-08-20T22:08:07.7858603Z { 2024-08-20T22:08:07.7859125Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7859773Z "size": 106, 2024-08-20T22:08:07.7860416Z "digest": "sha256:5645520256fa6d7f2270608eaed53849a131d03fbf4c5d4472c9cfde1a439e47" 2024-08-20T22:08:07.7861104Z }, 2024-08-20T22:08:07.7861496Z { 2024-08-20T22:08:07.7861994Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7862782Z "size": 503, 2024-08-20T22:08:07.7863445Z "digest": "sha256:829ed7d91e3818d0e11834c71551117c7aeec0a51d798e93d641faa0c02582cb" 2024-08-20T22:08:07.7864148Z }, 2024-08-20T22:08:07.7864447Z { 2024-08-20T22:08:07.7865113Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7865754Z "size": 121477267, 2024-08-20T22:08:07.7866347Z "digest": "sha256:385d0421d67c533d628985c453a186028a61f6ad4209ae519ec3d4ecea661d3e" 2024-08-20T22:08:07.7867110Z }, 2024-08-20T22:08:07.7867447Z { 2024-08-20T22:08:07.7867989Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7868691Z "size": 109, 2024-08-20T22:08:07.7869321Z "digest": "sha256:7cbfce999d76851e07dd71056a6d852629d789ccfd46be44cc405b451fb8bb58" 2024-08-20T22:08:07.7870027Z }, 2024-08-20T22:08:07.7870445Z { 2024-08-20T22:08:07.7870929Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7871667Z "size": 490, 2024-08-20T22:08:07.7872338Z "digest": "sha256:6658e558ebe496f0d628947534b0ea1e44808f84f2dfd8a9a704e8aa5329d791" 2024-08-20T22:08:07.7872985Z }, 2024-08-20T22:08:07.7873346Z { 2024-08-20T22:08:07.7873912Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7874549Z "size": 297, 2024-08-20T22:08:07.7875151Z "digest": "sha256:33444a5a7e01a3010fd1cc16b32a66892a08600d7f700747d23faf3c36624f3a" 2024-08-20T22:08:07.7875905Z }, 2024-08-20T22:08:07.7876374Z { 2024-08-20T22:08:07.7876961Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7877574Z "size": 103, 2024-08-20T22:08:07.7878304Z "digest": "sha256:7c47122f4e09b47e891b4ae9e499a35a4eb46faacae87d0e946615adda6e0706" 2024-08-20T22:08:07.7879060Z }, 2024-08-20T22:08:07.7879370Z { 2024-08-20T22:08:07.7879904Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7880591Z "size": 1473, 2024-08-20T22:08:07.7881242Z "digest": "sha256:1811ff4d3556ac255604f346d879528aff2aadd7652c2d39baf4fe6eaa60aec3" 2024-08-20T22:08:07.7881902Z }, 2024-08-20T22:08:07.7882293Z { 2024-08-20T22:08:07.7882839Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7883482Z "size": 594729516, 2024-08-20T22:08:07.7884132Z "digest": "sha256:7754ee068b33f760d9c320c26a1a3935247ace0c8c463a976105c1586a43fb1c" 2024-08-20T22:08:07.7884839Z }, 2024-08-20T22:08:07.7885255Z { 2024-08-20T22:08:07.7885835Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7886564Z "size": 164, 2024-08-20T22:08:07.7887193Z "digest": "sha256:0479fbac3361713c075bf788e778117917dcacdd1ff71ed79a789dd2016a1714" 2024-08-20T22:08:07.7888023Z }, 2024-08-20T22:08:07.7888330Z { 2024-08-20T22:08:07.7888857Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7889580Z "size": 564, 2024-08-20T22:08:07.7890200Z "digest": "sha256:f1d6874dee9c34d154278d2b6935d04447f07fa89d25d4a09398234040d8a7e8" 2024-08-20T22:08:07.7890855Z }, 2024-08-20T22:08:07.7891276Z { 2024-08-20T22:08:07.7891876Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7892541Z "size": 35874203, 2024-08-20T22:08:07.7893240Z "digest": "sha256:afc429e87b593a64f3e0a1be51e09e966ea1aa71e6e8a0bec4edc9f2f7bc1c05" 2024-08-20T22:08:07.7893953Z }, 2024-08-20T22:08:07.7894322Z { 2024-08-20T22:08:07.7894868Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7895510Z "size": 104, 2024-08-20T22:08:07.7896154Z "digest": "sha256:019620471c2592cd22437fe8ed1a94c784114cf7445c5c2ab36dd0bae727fc66" 2024-08-20T22:08:07.7896897Z }, 2024-08-20T22:08:07.7897200Z { 2024-08-20T22:08:07.7897739Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7898557Z "size": 425, 2024-08-20T22:08:07.7899220Z "digest": "sha256:27aeb1b2cd5e28d8f0e54fc1090baf88abc29ebf5fce088b493fcae99c9550d1" 2024-08-20T22:08:07.7899896Z }, 2024-08-20T22:08:07.7900288Z { 2024-08-20T22:08:07.7900829Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7901507Z "size": 20262183, 2024-08-20T22:08:07.7902104Z "digest": "sha256:829c0e19f4b7797e9120a86172fabfd710f9b3971ffd714a616931a3fadc7192" 2024-08-20T22:08:07.7902902Z }, 2024-08-20T22:08:07.7903284Z { 2024-08-20T22:08:07.7903769Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7904432Z "size": 438, 2024-08-20T22:08:07.7905096Z "digest": "sha256:560cd6122a3868e7e82e75eb103d182edf6512114d61ccdfae099af00cb16b4c" 2024-08-20T22:08:07.7905897Z }, 2024-08-20T22:08:07.7906205Z { 2024-08-20T22:08:07.7906853Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7907508Z "size": 701, 2024-08-20T22:08:07.7908121Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:07.7908821Z }, 2024-08-20T22:08:07.7909179Z { 2024-08-20T22:08:07.7909699Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7910382Z "size": 141, 2024-08-20T22:08:07.7911019Z "digest": "sha256:5a4eb19d72198a01f3babd4711c40fcf7c6e5cb9d5cfedc507b0061408d47097" 2024-08-20T22:08:07.7911732Z }, 2024-08-20T22:08:07.7912244Z { 2024-08-20T22:08:07.7912725Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7913369Z "size": 135, 2024-08-20T22:08:07.7914169Z "digest": "sha256:da6a838a66c72b5de9a5590c233779880ea901c1264940a1c618825d2501ffd1" 2024-08-20T22:08:07.7914873Z }, 2024-08-20T22:08:07.7915195Z { 2024-08-20T22:08:07.7915761Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7916403Z "size": 4838068375, 2024-08-20T22:08:07.7917067Z "digest": "sha256:cbb79a7053537bed2d80b77198ea552b985563b11d3f8616593c6ab575a3a090" 2024-08-20T22:08:07.7917772Z }, 2024-08-20T22:08:07.7918106Z { 2024-08-20T22:08:07.7918720Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7919366Z "size": 194, 2024-08-20T22:08:07.7920011Z "digest": "sha256:4d4fa6802ab3e6b36d8d4adb650046e9370c4a16a19876970559ef8a312ca855" 2024-08-20T22:08:07.7920699Z }, 2024-08-20T22:08:07.7921083Z { 2024-08-20T22:08:07.7921582Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7922211Z "size": 564, 2024-08-20T22:08:07.7922887Z "digest": "sha256:0f085d4fa279366fcd52d303b7ad8abddc5962a8872174b8109f3f5d5ca5f2cf" 2024-08-20T22:08:07.7923695Z }, 2024-08-20T22:08:07.7923997Z { 2024-08-20T22:08:07.7924591Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7925305Z "size": 43162921, 2024-08-20T22:08:07.7925985Z "digest": "sha256:4a0412373305517489d891a4749e6d2b80b4aeb7df3b8adca0ed0abb2f3e5300" 2024-08-20T22:08:07.7926665Z }, 2024-08-20T22:08:07.7927003Z { 2024-08-20T22:08:07.7927567Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7928243Z "size": 106, 2024-08-20T22:08:07.7928855Z "digest": "sha256:302c8c1779444e54919f83e91c6d5e1fdea653fc899be802b3a2a34d34d99d5d" 2024-08-20T22:08:07.7929617Z }, 2024-08-20T22:08:07.7929955Z { 2024-08-20T22:08:07.7930437Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7931143Z "size": 1380, 2024-08-20T22:08:07.7931759Z "digest": "sha256:02970347dff622ee45629bc28788716d00528464f71f88ec7b74ae5f44a486e2" 2024-08-20T22:08:07.7932513Z }, 2024-08-20T22:08:07.7932893Z { 2024-08-20T22:08:07.7933407Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7934047Z "size": 701, 2024-08-20T22:08:07.7934778Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:07.7935480Z }, 2024-08-20T22:08:07.7935823Z { 2024-08-20T22:08:07.7936662Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7937270Z "size": 139, 2024-08-20T22:08:07.7937923Z "digest": "sha256:916c5e8580164926a3aec87e88a5d1c2d02756e49fdeed4198e1e6304efa2fd3" 2024-08-20T22:08:07.7938707Z }, 2024-08-20T22:08:07.7939141Z { 2024-08-20T22:08:07.7939618Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7940298Z "size": 118, 2024-08-20T22:08:07.7940937Z "digest": "sha256:c42b3602a1fb4df5111224650f15adffb025979ae9c614807379ea84e189533b" 2024-08-20T22:08:07.7941686Z }, 2024-08-20T22:08:07.7941982Z { 2024-08-20T22:08:07.7942619Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7943317Z "size": 4324672728, 2024-08-20T22:08:07.7943936Z "digest": "sha256:11fad972ef6cb9d2dc236504b0ee5b6e5ea996e4576ece0ed3262fbb9ee2de60" 2024-08-20T22:08:07.7944668Z }, 2024-08-20T22:08:07.7945052Z { 2024-08-20T22:08:07.7945584Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7946252Z "size": 174, 2024-08-20T22:08:07.7946907Z "digest": "sha256:f957c16244819132b478272aaf24231699c4fcbd495c229a574f94daab8d6b86" 2024-08-20T22:08:07.7947613Z }, 2024-08-20T22:08:07.7947944Z { 2024-08-20T22:08:07.7948465Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7949120Z "size": 907, 2024-08-20T22:08:07.7949750Z "digest": "sha256:768c31dca016b80bbdc973c8ee818034bdbd17b7ea9bdadbf0ec45ea51005720" 2024-08-20T22:08:07.7950501Z }, 2024-08-20T22:08:07.7950967Z { 2024-08-20T22:08:07.7951486Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7952213Z "size": 701, 2024-08-20T22:08:07.7952931Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:07.7953703Z }, 2024-08-20T22:08:07.7954127Z { 2024-08-20T22:08:07.7954650Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7955250Z "size": 134, 2024-08-20T22:08:07.7955954Z "digest": "sha256:8c32d0ecebbe2054c96a3a5c1f3d6c8b87c2e826686b369e5242cd55d0f01c2d" 2024-08-20T22:08:07.7956664Z }, 2024-08-20T22:08:07.7957036Z { 2024-08-20T22:08:07.7957568Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7958209Z "size": 32, 2024-08-20T22:08:07.7958858Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:07.7959593Z }, 2024-08-20T22:08:07.7959993Z { 2024-08-20T22:08:07.7960542Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7961360Z "size": 158, 2024-08-20T22:08:07.7961973Z "digest": "sha256:78ee77eddef23f20afc78e2924f4467041cadb754634ee3df431c68211cf676d" 2024-08-20T22:08:07.7962676Z }, 2024-08-20T22:08:07.7963065Z { 2024-08-20T22:08:07.7963611Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7964212Z "size": 1841, 2024-08-20T22:08:07.7964894Z "digest": "sha256:37513e9b43d7123e7ddee09100f62a72b57af7b3833eaecd515e247e2f63e13f" 2024-08-20T22:08:07.7965607Z }, 2024-08-20T22:08:07.7965921Z { 2024-08-20T22:08:07.7966590Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7967304Z "size": 7529776, 2024-08-20T22:08:07.7968098Z "digest": "sha256:a7a5392e2bf6b73d9be292cc22a99ad4637636c15733beb7afbe6403b5ba21a8" 2024-08-20T22:08:07.7968862Z }, 2024-08-20T22:08:07.7969210Z { 2024-08-20T22:08:07.7969811Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7970465Z "size": 163, 2024-08-20T22:08:07.7971048Z "digest": "sha256:b10de02255c021b2366b85bf12ddf593aac89f88964201c455b71a145bc5ca2e" 2024-08-20T22:08:07.7971789Z }, 2024-08-20T22:08:07.7972150Z { 2024-08-20T22:08:07.7972667Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7973365Z "size": 7943, 2024-08-20T22:08:07.7974014Z "digest": "sha256:e56eeff2db93ffe5002a2bb764795b7333321c6802cc28bef0801a619fe6a07e" 2024-08-20T22:08:07.7974717Z }, 2024-08-20T22:08:07.7975086Z { 2024-08-20T22:08:07.7975608Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7976246Z "size": 8068, 2024-08-20T22:08:07.7976944Z "digest": "sha256:70ac3e38d0fd1f0f46ad710c3fc485fd01406606054e8cf84a63af1570bc9f3a" 2024-08-20T22:08:07.7977605Z }, 2024-08-20T22:08:07.7977941Z { 2024-08-20T22:08:07.7978540Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7979186Z "size": 302, 2024-08-20T22:08:07.7979765Z "digest": "sha256:eb09cbf3e4168749cc8605e3188ed35070f6037d13930e443c38d8941734488f" 2024-08-20T22:08:07.7980598Z }, 2024-08-20T22:08:07.7980948Z { 2024-08-20T22:08:07.7981452Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7982150Z "size": 7629638, 2024-08-20T22:08:07.7982865Z "digest": "sha256:ee1273153675e6543b58f5be50321abee5e91f7351701d5d501a6837eaf676ef" 2024-08-20T22:08:07.7983580Z }, 2024-08-20T22:08:07.7983927Z { 2024-08-20T22:08:07.7984447Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7985099Z "size": 108, 2024-08-20T22:08:07.7985775Z "digest": "sha256:e22979f9e0eedfa16f98eb4fa26f30a50c237f5ffd2f6e5b898f2b908058496d" 2024-08-20T22:08:07.7986437Z }, 2024-08-20T22:08:07.7986848Z { 2024-08-20T22:08:07.7987414Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7988118Z "size": 54145661, 2024-08-20T22:08:07.7988831Z "digest": "sha256:6c1f0aabce7c84f9b0b9ab3b96325778dba1c1465b7463c5b4c6be1cf52b74db" 2024-08-20T22:08:07.7989585Z }, 2024-08-20T22:08:07.7989936Z { 2024-08-20T22:08:07.7990411Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7991106Z "size": 473, 2024-08-20T22:08:07.7991725Z "digest": "sha256:7ebe49c3a23e71b289ce012b48dad35791a2f2aa7fcee2326a64d763b0d14df8" 2024-08-20T22:08:07.7992473Z }, 2024-08-20T22:08:07.7992802Z { 2024-08-20T22:08:07.7993377Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7994075Z "size": 1374858766, 2024-08-20T22:08:07.7994737Z "digest": "sha256:5d115b29b5d53e5418762bc41ae975dd245bcffc3d058b8902c58146d505cb37" 2024-08-20T22:08:07.7995387Z }, 2024-08-20T22:08:07.7995777Z { 2024-08-20T22:08:07.7996320Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.7996957Z "size": 106, 2024-08-20T22:08:07.7997640Z "digest": "sha256:7f2d7a04fbe65bbe1e895138e0cfeeef53ad926c890bab8e7e132ad1e655e48f" 2024-08-20T22:08:07.7998446Z }, 2024-08-20T22:08:07.7998785Z { 2024-08-20T22:08:07.7999250Z + exit 0 2024-08-20T22:08:07.7999747Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.8000475Z "size": 559, 2024-08-20T22:08:07.8001170Z "digest": "sha256:d79fd7d526989e554455eda7d459f2c88444030f61710101e52ec8c7a49b7b79" 2024-08-20T22:08:07.8001854Z }, 2024-08-20T22:08:07.8002182Z { 2024-08-20T22:08:07.8002754Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.8003398Z "size": 46248488, 2024-08-20T22:08:07.8004025Z "digest": "sha256:9752d6fbbcab4e7403f26777c5b65911320de6389c81b833a47ba45c4f3354b4" 2024-08-20T22:08:07.8004794Z }, 2024-08-20T22:08:07.8005136Z { 2024-08-20T22:08:07.8005673Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.8006336Z "size": 111, 2024-08-20T22:08:07.8007053Z "digest": "sha256:83808b2eff4d71ed46db7c29f66fe918f328405c25b314225dfad01b4868d1f4" 2024-08-20T22:08:07.8007750Z }, 2024-08-20T22:08:07.8008177Z { 2024-08-20T22:08:07.8008672Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.8009294Z "size": 32, 2024-08-20T22:08:07.8009956Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:07.8010629Z }, 2024-08-20T22:08:07.8010958Z { 2024-08-20T22:08:07.8011514Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.8012165Z "size": 32, 2024-08-20T22:08:07.8012745Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:07.8013505Z }, 2024-08-20T22:08:07.8013902Z { 2024-08-20T22:08:07.8014459Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.8015071Z "size": 32, 2024-08-20T22:08:07.8015685Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:07.8016430Z }, 2024-08-20T22:08:07.8016745Z { 2024-08-20T22:08:07.8017261Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:07.8017959Z "size": 32, 2024-08-20T22:08:07.8018617Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:07.8019273Z } 2024-08-20T22:08:07.8019672Z ] 2024-08-20T22:08:07.8020064Z } 2024-08-20T22:08:07.8326127Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:08:07.8326630Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:08:07.8327188Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T22:08:07.8336716Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:07.8337230Z env: 2024-08-20T22:08:07.8337515Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:07.8338730Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.8339903Z ##[endgroup] 2024-08-20T22:08:07.8374594Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.8428668Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T22:08:07.8429269Z with: 2024-08-20T22:08:07.8430353Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.8431665Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:07.8432198Z env: 2024-08-20T22:08:07.8432488Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:07.8432845Z ##[endgroup] 2024-08-20T22:08:07.8451234Z ##[group]Run set -x 2024-08-20T22:08:07.8451591Z set -x 2024-08-20T22:08:07.8451900Z set +e 2024-08-20T22:08:07.8452207Z  2024-08-20T22:08:07.8452489Z login() { 2024-08-20T22:08:07.8453152Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:08:07.8454047Z } 2024-08-20T22:08:07.8454333Z  2024-08-20T22:08:07.8454657Z retry () { 2024-08-20T22:08:07.8455059Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:08:07.8455510Z } 2024-08-20T22:08:07.8455790Z  2024-08-20T22:08:07.8456113Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:08:07.8456531Z  2024-08-20T22:08:07.8456809Z set -e 2024-08-20T22:08:07.8457288Z # ignore output since only exit code is used for conditional 2024-08-20T22:08:07.8457995Z # only pull docker image if it's not available locally 2024-08-20T22:08:07.8458757Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T22:08:07.8459466Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T22:08:07.8459916Z fi 2024-08-20T22:08:07.8468921Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:07.8469433Z env: 2024-08-20T22:08:07.8469723Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:07.8470885Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:07.8472194Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:07.8472719Z ##[endgroup] 2024-08-20T22:08:07.8502773Z + set +e 2024-08-20T22:08:07.8503338Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:07.8503988Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:07.8506793Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:08:07.8508251Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:08.3806326Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:08:08.3807178Z Configure a credential helper to remove this warning. See 2024-08-20T22:08:08.3808203Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:08:08.3808772Z 2024-08-20T22:08:08.3808910Z Login Succeeded 2024-08-20T22:08:08.3830002Z + set -e 2024-08-20T22:08:08.3831309Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:08.3990342Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:08.3992500Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:08.6576239Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-08-20T22:08:08.6585405Z 7a2c55901189: Pulling fs layer 2024-08-20T22:08:08.6586660Z 224fe954d725: Pulling fs layer 2024-08-20T22:08:08.6587846Z 75722010b82e: Pulling fs layer 2024-08-20T22:08:08.6588516Z d527cbbb87e3: Pulling fs layer 2024-08-20T22:08:08.6589019Z b57676e46aee: Pulling fs layer 2024-08-20T22:08:08.6589483Z a8c1e85b5e14: Pulling fs layer 2024-08-20T22:08:08.6589988Z a41a8d1c11c8: Pulling fs layer 2024-08-20T22:08:08.6590623Z 0c1227890755: Pulling fs layer 2024-08-20T22:08:08.6591279Z d8d1234baab3: Pulling fs layer 2024-08-20T22:08:08.6591841Z 7ed32bc8e469: Pulling fs layer 2024-08-20T22:08:08.6592389Z ec1e7978c1fe: Pulling fs layer 2024-08-20T22:08:08.6592904Z ac23c5fcc1ef: Pulling fs layer 2024-08-20T22:08:08.6593399Z ca426ffc9d2b: Pulling fs layer 2024-08-20T22:08:08.6593831Z 4373aa743caf: Pulling fs layer 2024-08-20T22:08:08.6594227Z 75b2abb16c68: Pulling fs layer 2024-08-20T22:08:08.6594601Z b57676e46aee: Waiting 2024-08-20T22:08:08.6595019Z ef1e3ef1e85f: Pulling fs layer 2024-08-20T22:08:08.6595825Z d527cbbb87e3: Waiting 2024-08-20T22:08:08.6596283Z 0c1227890755: Waiting 2024-08-20T22:08:08.6596731Z a8c1e85b5e14: Waiting 2024-08-20T22:08:08.6597183Z ac23c5fcc1ef: Waiting 2024-08-20T22:08:08.6597637Z d8d1234baab3: Waiting 2024-08-20T22:08:08.6598102Z 0bf61114d37b: Pulling fs layer 2024-08-20T22:08:08.6598621Z 2df849f72ba4: Pulling fs layer 2024-08-20T22:08:08.6599145Z 4373aa743caf: Waiting 2024-08-20T22:08:08.6599627Z 34c47d63c5de: Pulling fs layer 2024-08-20T22:08:08.6600118Z a41a8d1c11c8: Waiting 2024-08-20T22:08:08.6600585Z 1fef4bd19792: Pulling fs layer 2024-08-20T22:08:08.6601069Z 75b2abb16c68: Waiting 2024-08-20T22:08:08.6601505Z 0bf61114d37b: Waiting 2024-08-20T22:08:08.6601940Z 34c47d63c5de: Waiting 2024-08-20T22:08:08.6602379Z 6efb96fc87ea: Pulling fs layer 2024-08-20T22:08:08.6602881Z b119c4fc6c58: Pulling fs layer 2024-08-20T22:08:08.6603398Z 33b639226eee: Pulling fs layer 2024-08-20T22:08:08.6603901Z d3547bd43146: Pulling fs layer 2024-08-20T22:08:08.6604388Z 6efb96fc87ea: Waiting 2024-08-20T22:08:08.6604827Z b119c4fc6c58: Waiting 2024-08-20T22:08:08.6605288Z 5645520256fa: Pulling fs layer 2024-08-20T22:08:08.6605820Z 829ed7d91e38: Pulling fs layer 2024-08-20T22:08:08.6606301Z 385d0421d67c: Pulling fs layer 2024-08-20T22:08:08.6606750Z 33b639226eee: Waiting 2024-08-20T22:08:08.6607200Z 7cbfce999d76: Pulling fs layer 2024-08-20T22:08:08.6607699Z 6658e558ebe4: Pulling fs layer 2024-08-20T22:08:08.6608194Z 829ed7d91e38: Waiting 2024-08-20T22:08:08.6608631Z 5645520256fa: Waiting 2024-08-20T22:08:08.6609050Z 33444a5a7e01: Pulling fs layer 2024-08-20T22:08:08.6609506Z d3547bd43146: Waiting 2024-08-20T22:08:08.6609909Z 6658e558ebe4: Waiting 2024-08-20T22:08:08.6610332Z 7c47122f4e09: Pulling fs layer 2024-08-20T22:08:08.6610873Z 1811ff4d3556: Pulling fs layer 2024-08-20T22:08:08.6611324Z 7c47122f4e09: Waiting 2024-08-20T22:08:08.6611735Z 7754ee068b33: Pulling fs layer 2024-08-20T22:08:08.6612221Z 0479fbac3361: Pulling fs layer 2024-08-20T22:08:08.6612705Z f1d6874dee9c: Pulling fs layer 2024-08-20T22:08:08.6613167Z 1811ff4d3556: Waiting 2024-08-20T22:08:08.6613587Z afc429e87b59: Pulling fs layer 2024-08-20T22:08:08.6614052Z 0479fbac3361: Waiting 2024-08-20T22:08:08.6614456Z 7cbfce999d76: Waiting 2024-08-20T22:08:08.6614880Z 019620471c25: Pulling fs layer 2024-08-20T22:08:08.6615385Z 27aeb1b2cd5e: Pulling fs layer 2024-08-20T22:08:08.6615858Z afc429e87b59: Waiting 2024-08-20T22:08:08.6616265Z 829c0e19f4b7: Pulling fs layer 2024-08-20T22:08:08.6616720Z f1d6874dee9c: Waiting 2024-08-20T22:08:08.6617134Z 560cd6122a38: Pulling fs layer 2024-08-20T22:08:08.6617584Z 27aeb1b2cd5e: Waiting 2024-08-20T22:08:08.6617976Z 829c0e19f4b7: Waiting 2024-08-20T22:08:08.6618415Z 019620471c25: Waiting 2024-08-20T22:08:08.6618798Z 560cd6122a38: Waiting 2024-08-20T22:08:08.6619211Z 5a4eb19d7219: Pulling fs layer 2024-08-20T22:08:08.6619693Z da6a838a66c7: Pulling fs layer 2024-08-20T22:08:08.6620172Z cbb79a705353: Pulling fs layer 2024-08-20T22:08:08.6620655Z 4d4fa6802ab3: Pulling fs layer 2024-08-20T22:08:08.6621170Z 0f085d4fa279: Pulling fs layer 2024-08-20T22:08:08.6621606Z da6a838a66c7: Waiting 2024-08-20T22:08:08.6622029Z cbb79a705353: Waiting 2024-08-20T22:08:08.6622887Z 4a0412373305: Pulling fs layer 2024-08-20T22:08:08.6623301Z 5a4eb19d7219: Waiting 2024-08-20T22:08:08.6623704Z 302c8c177944: Pulling fs layer 2024-08-20T22:08:08.6624176Z 4a0412373305: Waiting 2024-08-20T22:08:08.6624601Z 02970347dff6: Pulling fs layer 2024-08-20T22:08:08.6625097Z 916c5e858016: Pulling fs layer 2024-08-20T22:08:08.6625568Z c42b3602a1fb: Pulling fs layer 2024-08-20T22:08:08.6626076Z 02970347dff6: Waiting 2024-08-20T22:08:08.6626525Z 11fad972ef6c: Pulling fs layer 2024-08-20T22:08:08.6626978Z 302c8c177944: Waiting 2024-08-20T22:08:08.6627405Z f957c1624481: Pulling fs layer 2024-08-20T22:08:08.6627865Z 0f085d4fa279: Waiting 2024-08-20T22:08:08.6628255Z 916c5e858016: Waiting 2024-08-20T22:08:08.6628573Z 11fad972ef6c: Waiting 2024-08-20T22:08:08.6628912Z 768c31dca016: Pulling fs layer 2024-08-20T22:08:08.6629263Z f957c1624481: Waiting 2024-08-20T22:08:08.6629746Z c42b3602a1fb: Waiting 2024-08-20T22:08:08.6632194Z 768c31dca016: Waiting 2024-08-20T22:08:08.6632840Z 8c32d0ecebbe: Pulling fs layer 2024-08-20T22:08:08.6633385Z 4f4fb700ef54: Pulling fs layer 2024-08-20T22:08:08.6633892Z 78ee77eddef2: Pulling fs layer 2024-08-20T22:08:08.6634382Z 8c32d0ecebbe: Waiting 2024-08-20T22:08:08.6634733Z 37513e9b43d7: Pulling fs layer 2024-08-20T22:08:08.6635092Z 37513e9b43d7: Waiting 2024-08-20T22:08:08.6635433Z a7a5392e2bf6: Pulling fs layer 2024-08-20T22:08:08.6635816Z b10de02255c0: Pulling fs layer 2024-08-20T22:08:08.6636207Z e56eeff2db93: Pulling fs layer 2024-08-20T22:08:08.6636882Z a7a5392e2bf6: Waiting 2024-08-20T22:08:08.6637219Z 70ac3e38d0fd: Pulling fs layer 2024-08-20T22:08:08.6637595Z eb09cbf3e416: Pulling fs layer 2024-08-20T22:08:08.6637956Z b10de02255c0: Waiting 2024-08-20T22:08:08.6638303Z e56eeff2db93: Waiting 2024-08-20T22:08:08.6638641Z ee1273153675: Pulling fs layer 2024-08-20T22:08:08.6639012Z e22979f9e0ee: Pulling fs layer 2024-08-20T22:08:08.6639379Z e22979f9e0ee: Waiting 2024-08-20T22:08:08.6639715Z 6c1f0aabce7c: Pulling fs layer 2024-08-20T22:08:08.6640093Z 7ebe49c3a23e: Pulling fs layer 2024-08-20T22:08:08.6640454Z 6c1f0aabce7c: Waiting 2024-08-20T22:08:08.6640768Z 7ebe49c3a23e: Waiting 2024-08-20T22:08:08.6641095Z 5d115b29b5d5: Pulling fs layer 2024-08-20T22:08:08.6641469Z 7f2d7a04fbe6: Pulling fs layer 2024-08-20T22:08:08.6641836Z d79fd7d52698: Pulling fs layer 2024-08-20T22:08:08.6642210Z 9752d6fbbcab: Pulling fs layer 2024-08-20T22:08:08.6642578Z 83808b2eff4d: Pulling fs layer 2024-08-20T22:08:08.6642937Z 7f2d7a04fbe6: Waiting 2024-08-20T22:08:08.6643245Z 83808b2eff4d: Waiting 2024-08-20T22:08:08.6643556Z 9752d6fbbcab: Waiting 2024-08-20T22:08:08.6643873Z d79fd7d52698: Waiting 2024-08-20T22:08:08.7874602Z 224fe954d725: Verifying Checksum 2024-08-20T22:08:08.7875139Z 224fe954d725: Download complete 2024-08-20T22:08:08.8807033Z d527cbbb87e3: Download complete 2024-08-20T22:08:08.9474946Z b57676e46aee: Verifying Checksum 2024-08-20T22:08:08.9475580Z b57676e46aee: Download complete 2024-08-20T22:08:08.9998492Z 7a2c55901189: Download complete 2024-08-20T22:08:09.0758052Z a41a8d1c11c8: Download complete 2024-08-20T22:08:09.1411869Z 0c1227890755: Download complete 2024-08-20T22:08:09.2217756Z d8d1234baab3: Download complete 2024-08-20T22:08:09.2686660Z 75722010b82e: Verifying Checksum 2024-08-20T22:08:09.2687214Z 75722010b82e: Download complete 2024-08-20T22:08:09.3384529Z ec1e7978c1fe: Verifying Checksum 2024-08-20T22:08:09.3385037Z ec1e7978c1fe: Download complete 2024-08-20T22:08:09.4054411Z ac23c5fcc1ef: Download complete 2024-08-20T22:08:10.1707622Z 7a2c55901189: Pull complete 2024-08-20T22:08:10.4902925Z 224fe954d725: Pull complete 2024-08-20T22:08:11.4023892Z 75722010b82e: Pull complete 2024-08-20T22:08:11.4160519Z d527cbbb87e3: Pull complete 2024-08-20T22:08:11.4295514Z b57676e46aee: Pull complete 2024-08-20T22:08:11.9175683Z ca426ffc9d2b: Verifying Checksum 2024-08-20T22:08:11.9176195Z ca426ffc9d2b: Download complete 2024-08-20T22:08:11.9864587Z 4373aa743caf: Verifying Checksum 2024-08-20T22:08:11.9865044Z 4373aa743caf: Download complete 2024-08-20T22:08:12.0616618Z 75b2abb16c68: Download complete 2024-08-20T22:08:12.1423432Z ef1e3ef1e85f: Verifying Checksum 2024-08-20T22:08:12.1423978Z ef1e3ef1e85f: Download complete 2024-08-20T22:08:13.1130353Z 0bf61114d37b: Download complete 2024-08-20T22:08:13.1847704Z 2df849f72ba4: Verifying Checksum 2024-08-20T22:08:13.1848181Z 2df849f72ba4: Download complete 2024-08-20T22:08:13.2609049Z 34c47d63c5de: Verifying Checksum 2024-08-20T22:08:13.2609503Z 34c47d63c5de: Download complete 2024-08-20T22:08:13.3260934Z 1fef4bd19792: Verifying Checksum 2024-08-20T22:08:13.3261445Z 1fef4bd19792: Download complete 2024-08-20T22:08:22.2948586Z a8c1e85b5e14: Verifying Checksum 2024-08-20T22:08:22.2949072Z a8c1e85b5e14: Download complete 2024-08-20T22:08:22.3671378Z b119c4fc6c58: Verifying Checksum 2024-08-20T22:08:22.3671864Z b119c4fc6c58: Download complete 2024-08-20T22:08:22.4295530Z 33b639226eee: Download complete 2024-08-20T22:08:22.5057514Z d3547bd43146: Verifying Checksum 2024-08-20T22:08:22.5057966Z d3547bd43146: Download complete 2024-08-20T22:08:22.5825155Z 5645520256fa: Verifying Checksum 2024-08-20T22:08:22.5825655Z 5645520256fa: Download complete 2024-08-20T22:08:22.6596586Z 829ed7d91e38: Download complete 2024-08-20T22:08:23.9292495Z 385d0421d67c: Verifying Checksum 2024-08-20T22:08:23.9292988Z 385d0421d67c: Download complete 2024-08-20T22:08:24.0056078Z 7cbfce999d76: Download complete 2024-08-20T22:08:24.0769015Z 6658e558ebe4: Verifying Checksum 2024-08-20T22:08:24.0769444Z 6658e558ebe4: Download complete 2024-08-20T22:08:24.1388244Z 33444a5a7e01: Verifying Checksum 2024-08-20T22:08:24.1388677Z 33444a5a7e01: Download complete 2024-08-20T22:08:24.2121890Z 7c47122f4e09: Verifying Checksum 2024-08-20T22:08:24.2122455Z 7c47122f4e09: Download complete 2024-08-20T22:08:24.3599952Z 1811ff4d3556: Verifying Checksum 2024-08-20T22:08:24.3600627Z 1811ff4d3556: Download complete 2024-08-20T22:08:30.3518228Z 7754ee068b33: Verifying Checksum 2024-08-20T22:08:30.3518719Z 7754ee068b33: Download complete 2024-08-20T22:08:30.4212298Z 0479fbac3361: Verifying Checksum 2024-08-20T22:08:30.4212829Z 0479fbac3361: Download complete 2024-08-20T22:08:30.4918892Z f1d6874dee9c: Download complete 2024-08-20T22:08:30.8909577Z afc429e87b59: Verifying Checksum 2024-08-20T22:08:30.8910157Z afc429e87b59: Download complete 2024-08-20T22:08:30.9640041Z 019620471c25: Download complete 2024-08-20T22:08:31.0265388Z 27aeb1b2cd5e: Download complete 2024-08-20T22:08:31.2819532Z 829c0e19f4b7: Verifying Checksum 2024-08-20T22:08:31.2820152Z 829c0e19f4b7: Download complete 2024-08-20T22:08:31.3600533Z 560cd6122a38: Verifying Checksum 2024-08-20T22:08:31.3601090Z 560cd6122a38: Download complete 2024-08-20T22:08:31.4360759Z 5a4eb19d7219: Verifying Checksum 2024-08-20T22:08:31.4361240Z 5a4eb19d7219: Download complete 2024-08-20T22:08:31.4997472Z da6a838a66c7: Verifying Checksum 2024-08-20T22:08:31.4997929Z da6a838a66c7: Download complete 2024-08-20T22:08:34.5513299Z 7ed32bc8e469: Download complete 2024-08-20T22:08:34.6132398Z 4d4fa6802ab3: Verifying Checksum 2024-08-20T22:08:34.6133160Z 4d4fa6802ab3: Download complete 2024-08-20T22:08:34.6730663Z 0f085d4fa279: Verifying Checksum 2024-08-20T22:08:34.6731208Z 0f085d4fa279: Download complete 2024-08-20T22:08:35.1600397Z 4a0412373305: Verifying Checksum 2024-08-20T22:08:35.1600979Z 4a0412373305: Download complete 2024-08-20T22:08:35.2480255Z 302c8c177944: Verifying Checksum 2024-08-20T22:08:35.2481220Z 302c8c177944: Download complete 2024-08-20T22:08:35.3247850Z 02970347dff6: Verifying Checksum 2024-08-20T22:08:35.3248450Z 02970347dff6: Download complete 2024-08-20T22:08:35.4026231Z 916c5e858016: Verifying Checksum 2024-08-20T22:08:35.4026776Z 916c5e858016: Download complete 2024-08-20T22:08:35.4799339Z c42b3602a1fb: Verifying Checksum 2024-08-20T22:08:35.4799899Z c42b3602a1fb: Download complete 2024-08-20T22:08:35.5820349Z a8c1e85b5e14: Pull complete 2024-08-20T22:08:35.7498636Z a41a8d1c11c8: Pull complete 2024-08-20T22:08:35.8540884Z 0c1227890755: Pull complete 2024-08-20T22:08:35.9339321Z d8d1234baab3: Pull complete 2024-08-20T22:08:42.0116656Z 6efb96fc87ea: Verifying Checksum 2024-08-20T22:08:42.0117326Z 6efb96fc87ea: Download complete 2024-08-20T22:08:42.0862827Z f957c1624481: Verifying Checksum 2024-08-20T22:08:42.0863462Z f957c1624481: Download complete 2024-08-20T22:08:42.1511162Z 768c31dca016: Verifying Checksum 2024-08-20T22:08:42.1511821Z 768c31dca016: Download complete 2024-08-20T22:08:42.2219143Z 8c32d0ecebbe: Verifying Checksum 2024-08-20T22:08:42.2306412Z 4f4fb700ef54: Verifying Checksum 2024-08-20T22:08:42.2306984Z 4f4fb700ef54: Download complete 2024-08-20T22:08:42.3016270Z 78ee77eddef2: Verifying Checksum 2024-08-20T22:08:42.3016887Z 78ee77eddef2: Download complete 2024-08-20T22:08:42.3813243Z 37513e9b43d7: Download complete 2024-08-20T22:08:42.5100983Z a7a5392e2bf6: Verifying Checksum 2024-08-20T22:08:42.5101598Z a7a5392e2bf6: Download complete 2024-08-20T22:08:42.5838899Z b10de02255c0: Verifying Checksum 2024-08-20T22:08:42.5839818Z b10de02255c0: Download complete 2024-08-20T22:08:42.6685720Z e56eeff2db93: Verifying Checksum 2024-08-20T22:08:42.6686278Z e56eeff2db93: Download complete 2024-08-20T22:08:42.7398959Z 70ac3e38d0fd: Verifying Checksum 2024-08-20T22:08:42.7400460Z 70ac3e38d0fd: Download complete 2024-08-20T22:08:42.9317251Z ee1273153675: Verifying Checksum 2024-08-20T22:08:42.9317812Z ee1273153675: Download complete 2024-08-20T22:08:43.0017023Z e22979f9e0ee: Download complete 2024-08-20T22:08:43.5981019Z 6c1f0aabce7c: Verifying Checksum 2024-08-20T22:08:43.5981577Z 6c1f0aabce7c: Download complete 2024-08-20T22:08:43.6812863Z 7ebe49c3a23e: Verifying Checksum 2024-08-20T22:08:43.6813336Z 7ebe49c3a23e: Download complete 2024-08-20T22:09:10.6445031Z 5d115b29b5d5: Verifying Checksum 2024-08-20T22:09:10.6447420Z 5d115b29b5d5: Download complete 2024-08-20T22:09:10.7281099Z 7f2d7a04fbe6: Verifying Checksum 2024-08-20T22:09:10.7281667Z 7f2d7a04fbe6: Download complete 2024-08-20T22:09:10.8024045Z d79fd7d52698: Download complete 2024-08-20T22:09:11.9886129Z 9752d6fbbcab: Verifying Checksum 2024-08-20T22:09:11.9889093Z 9752d6fbbcab: Download complete 2024-08-20T22:09:12.0583166Z 83808b2eff4d: Verifying Checksum 2024-08-20T22:09:12.0583623Z 83808b2eff4d: Download complete 2024-08-20T22:09:49.0026519Z 11fad972ef6c: Verifying Checksum 2024-08-20T22:09:49.0027002Z 11fad972ef6c: Download complete 2024-08-20T22:09:52.0814207Z cbb79a705353: Verifying Checksum 2024-08-20T22:09:52.0814787Z cbb79a705353: Download complete 2024-08-20T22:10:06.8521494Z 7ed32bc8e469: Pull complete 2024-08-20T22:10:07.0447600Z ec1e7978c1fe: Pull complete 2024-08-20T22:10:07.2438697Z ac23c5fcc1ef: Pull complete 2024-08-20T22:10:15.7698987Z ca426ffc9d2b: Pull complete 2024-08-20T22:10:15.9608672Z 4373aa743caf: Pull complete 2024-08-20T22:10:16.1605257Z 75b2abb16c68: Pull complete 2024-08-20T22:10:16.3132013Z ef1e3ef1e85f: Pull complete 2024-08-20T22:10:18.8933397Z 0bf61114d37b: Pull complete 2024-08-20T22:10:19.0907347Z 2df849f72ba4: Pull complete 2024-08-20T22:10:19.2870510Z 34c47d63c5de: Pull complete 2024-08-20T22:10:19.5178850Z 1fef4bd19792: Pull complete 2024-08-20T22:11:17.7313858Z 6efb96fc87ea: Pull complete 2024-08-20T22:11:17.8553640Z b119c4fc6c58: Pull complete 2024-08-20T22:11:17.9379146Z 33b639226eee: Pull complete 2024-08-20T22:11:17.9875789Z d3547bd43146: Pull complete 2024-08-20T22:11:18.0510180Z 5645520256fa: Pull complete 2024-08-20T22:11:18.1076717Z 829ed7d91e38: Pull complete 2024-08-20T22:11:21.4503267Z 385d0421d67c: Pull complete 2024-08-20T22:11:21.5976779Z 7cbfce999d76: Pull complete 2024-08-20T22:11:21.6886332Z 6658e558ebe4: Pull complete 2024-08-20T22:11:21.7445049Z 33444a5a7e01: Pull complete 2024-08-20T22:11:21.7905935Z 7c47122f4e09: Pull complete 2024-08-20T22:11:21.8410542Z 1811ff4d3556: Pull complete 2024-08-20T22:11:32.4247001Z 7754ee068b33: Pull complete 2024-08-20T22:11:32.5914018Z 0479fbac3361: Pull complete 2024-08-20T22:11:32.7719602Z f1d6874dee9c: Pull complete 2024-08-20T22:11:33.6710606Z afc429e87b59: Pull complete 2024-08-20T22:11:33.7529856Z 019620471c25: Pull complete 2024-08-20T22:11:33.9243239Z 27aeb1b2cd5e: Pull complete 2024-08-20T22:11:34.4246076Z 829c0e19f4b7: Pull complete 2024-08-20T22:11:34.6192756Z 560cd6122a38: Pull complete 2024-08-20T22:11:34.9554002Z 5a4eb19d7219: Pull complete 2024-08-20T22:11:35.0526712Z da6a838a66c7: Pull complete 2024-08-20T22:12:13.7857370Z cbb79a705353: Pull complete 2024-08-20T22:12:14.0012645Z 4d4fa6802ab3: Pull complete 2024-08-20T22:12:14.2084579Z 0f085d4fa279: Pull complete 2024-08-20T22:12:16.5573531Z 4a0412373305: Pull complete 2024-08-20T22:12:16.5757918Z 302c8c177944: Pull complete 2024-08-20T22:12:16.5887722Z 02970347dff6: Pull complete 2024-08-20T22:12:16.6522960Z 916c5e858016: Pull complete 2024-08-20T22:12:16.7324620Z c42b3602a1fb: Pull complete 2024-08-20T22:13:40.0443675Z 11fad972ef6c: Pull complete 2024-08-20T22:13:40.2343073Z f957c1624481: Pull complete 2024-08-20T22:13:40.3139475Z 768c31dca016: Pull complete 2024-08-20T22:13:40.6538651Z 8c32d0ecebbe: Pull complete 2024-08-20T22:13:40.7047920Z 4f4fb700ef54: Pull complete 2024-08-20T22:13:40.8396946Z 78ee77eddef2: Pull complete 2024-08-20T22:13:41.0416225Z 37513e9b43d7: Pull complete 2024-08-20T22:13:41.5344587Z a7a5392e2bf6: Pull complete 2024-08-20T22:13:41.6333544Z b10de02255c0: Pull complete 2024-08-20T22:13:41.8321948Z e56eeff2db93: Pull complete 2024-08-20T22:13:42.0344743Z 70ac3e38d0fd: Pull complete 2024-08-20T22:13:42.2490227Z eb09cbf3e416: Pull complete 2024-08-20T22:13:43.6547215Z ee1273153675: Pull complete 2024-08-20T22:13:43.8803970Z e22979f9e0ee: Pull complete 2024-08-20T22:13:46.5974896Z 6c1f0aabce7c: Pull complete 2024-08-20T22:13:46.8289869Z 7ebe49c3a23e: Pull complete 2024-08-20T22:14:36.2120582Z 5d115b29b5d5: Pull complete 2024-08-20T22:14:36.4263608Z 7f2d7a04fbe6: Pull complete 2024-08-20T22:14:36.6275715Z d79fd7d52698: Pull complete 2024-08-20T22:14:38.5872996Z 9752d6fbbcab: Pull complete 2024-08-20T22:14:38.7916412Z 83808b2eff4d: Pull complete 2024-08-20T22:14:39.5703617Z Digest: sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T22:14:39.6091876Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:14:39.6394745Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:14:39.6492876Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:14:39.6493801Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:14:39.6504284Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:14:39.6504790Z env: 2024-08-20T22:14:39.6505073Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:39.6505421Z ##[endgroup] 2024-08-20T22:14:39.6690407Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-20T22:14:39.6690971Z with: 2024-08-20T22:14:39.6691255Z driver-version: 550.54.15 2024-08-20T22:14:39.6691593Z env: 2024-08-20T22:14:39.6691873Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:39.6692218Z ##[endgroup] 2024-08-20T22:14:39.6809351Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:14:39.6809914Z with: 2024-08-20T22:14:39.6810198Z timeout_minutes: 10 2024-08-20T22:14:39.6810525Z max_attempts: 3 2024-08-20T22:14:39.6843000Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-20T22:14:39.6875473Z retry_wait_seconds: 10 2024-08-20T22:14:39.6875926Z polling_interval_seconds: 1 2024-08-20T22:14:39.6876301Z warning_on_retry: true 2024-08-20T22:14:39.6876659Z continue_on_error: false 2024-08-20T22:14:39.6877009Z env: 2024-08-20T22:14:39.6877293Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:39.6877647Z DRIVER_VERSION: 550.54.15 2024-08-20T22:14:39.6878001Z ##[endgroup] 2024-08-20T22:14:40.1749739Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-20T22:14:40.1750354Z + pre_install_nvidia_driver_amzn2 2024-08-20T22:14:40.1750876Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-20T22:14:42.7489596Z No match for argument: nvidia-driver-latest-dkms 2024-08-20T22:14:42.7490328Z No packages marked for removal. 2024-08-20T22:14:42.8015548Z Dependencies resolved. 2024-08-20T22:14:42.8028618Z Nothing to do. 2024-08-20T22:14:42.8028938Z Complete! 2024-08-20T22:14:43.0103784Z + install_nvidia_driver_common 2024-08-20T22:14:43.0107510Z + echo 'Before installing NVIDIA driver' 2024-08-20T22:14:43.0108863Z + lspci 2024-08-20T22:14:43.0109719Z Before installing NVIDIA driver 2024-08-20T22:14:43.0218783Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:14:43.0220402Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:14:43.0221358Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:14:43.0222143Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:14:43.0223050Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:14:43.0223835Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:14:43.0224559Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:14:43.0225355Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:14:43.0225926Z + lsmod 2024-08-20T22:14:43.0269080Z Module Size Used by 2024-08-20T22:14:43.0269666Z xt_nat 16384 0 2024-08-20T22:14:43.0270049Z ib_core 454656 0 2024-08-20T22:14:43.0270409Z veth 36864 0 2024-08-20T22:14:43.0270781Z nvidia_modeset 1351680 0 2024-08-20T22:14:43.0271197Z video 65536 1 nvidia_modeset 2024-08-20T22:14:43.0271633Z wmi 36864 1 video 2024-08-20T22:14:43.0272027Z nvidia_uvm 4706304 0 2024-08-20T22:14:43.0272473Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-20T22:14:43.0272952Z drm 602112 1 nvidia 2024-08-20T22:14:43.0273394Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:14:43.0273911Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-20T22:14:43.0274415Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:14:43.0274838Z xt_conntrack 16384 1 2024-08-20T22:14:43.0275309Z nft_chain_nat 16384 3 2024-08-20T22:14:43.0275738Z xt_MASQUERADE 20480 1 2024-08-20T22:14:43.0276208Z nf_nat 57344 3 xt_nat,nft_chain_nat,xt_MASQUERADE 2024-08-20T22:14:43.0276722Z nf_conntrack_netlink 57344 0 2024-08-20T22:14:43.0277330Z nf_conntrack 184320 5 xt_conntrack,nf_nat,xt_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:14:43.0277984Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:14:43.0278455Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:14:43.0278882Z xfrm_user 57344 1 2024-08-20T22:14:43.0279273Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:14:43.0281810Z xt_addrtype 16384 2 2024-08-20T22:14:43.0282376Z nft_compat 20480 4 2024-08-20T22:14:43.0283048Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:14:43.0283835Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:14:43.0284558Z br_netfilter 36864 0 2024-08-20T22:14:43.0285072Z bridge 307200 1 br_netfilter 2024-08-20T22:14:43.0285539Z stp 16384 1 bridge 2024-08-20T22:14:43.0286390Z llc 16384 2 bridge,stp 2024-08-20T22:14:43.0286943Z overlay 167936 0 2024-08-20T22:14:43.0287426Z tls 114688 0 2024-08-20T22:14:43.0287899Z nls_ascii 16384 1 2024-08-20T22:14:43.0288383Z nls_cp437 20480 1 2024-08-20T22:14:43.0288864Z vfat 24576 1 2024-08-20T22:14:43.0289341Z sunrpc 692224 1 2024-08-20T22:14:43.0289705Z fat 86016 1 vfat 2024-08-20T22:14:43.0290128Z ena 167936 0 2024-08-20T22:14:43.0290498Z ghash_clmulni_intel 16384 0 2024-08-20T22:14:43.0290877Z ptp 36864 1 ena 2024-08-20T22:14:43.0291271Z pps_core 24576 1 ptp 2024-08-20T22:14:43.0291660Z aesni_intel 393216 0 2024-08-20T22:14:43.0292129Z i8042 45056 0 2024-08-20T22:14:43.0292613Z serio 28672 3 i8042 2024-08-20T22:14:43.0293116Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:14:43.0293652Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:14:43.0294149Z button 24576 0 2024-08-20T22:14:43.0294521Z sch_fq_codel 20480 17 2024-08-20T22:14:43.0294889Z dm_mod 188416 0 2024-08-20T22:14:43.0295358Z fuse 163840 1 2024-08-20T22:14:43.0295834Z configfs 57344 1 2024-08-20T22:14:43.0296273Z dax 45056 1 dm_mod 2024-08-20T22:14:43.0296667Z loop 36864 0 2024-08-20T22:14:43.0297034Z dmi_sysfs 20480 0 2024-08-20T22:14:43.0297407Z crc32_pclmul 16384 0 2024-08-20T22:14:43.0297777Z crc32c_intel 24576 0 2024-08-20T22:14:43.0298139Z efivarfs 24576 1 2024-08-20T22:14:43.0298581Z + modinfo nvidia 2024-08-20T22:14:43.0299439Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:14:43.0300267Z alias: char-major-195-* 2024-08-20T22:14:43.0300658Z version: 550.54.15 2024-08-20T22:14:43.0301006Z supported: external 2024-08-20T22:14:43.0301347Z license: NVIDIA 2024-08-20T22:14:43.0301723Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:14:43.0302207Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:14:43.0302836Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:14:43.0303305Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:14:43.0303800Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:14:43.0304290Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:14:43.0304789Z depends: i2c-core,drm 2024-08-20T22:14:43.0305151Z retpoline: Y 2024-08-20T22:14:43.0305455Z name: nvidia 2024-08-20T22:14:43.0306070Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:14:43.0306768Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:14:43.0307417Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:14:43.0308015Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:14:43.0308458Z parm: NVreg_RmLogonRC:int 2024-08-20T22:14:43.0308893Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:14:43.0309344Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:14:43.0309779Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:14:43.0310217Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:14:43.0310748Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:14:43.0311303Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:14:43.0311781Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:14:43.0312212Z parm: NVreg_EnableMSI:int 2024-08-20T22:14:43.0312626Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:14:43.0313083Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:14:43.0313617Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:14:43.0314203Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:14:43.0314862Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:14:43.0315471Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:43.0316072Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:14:43.0316686Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:43.0317279Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:14:43.0317769Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:14:43.0318309Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:14:43.0318855Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:14:43.0319357Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:14:43.0319820Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:14:43.0320308Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:14:43.0320789Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:14:43.0321230Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:14:43.0321746Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:14:43.0322273Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:14:43.0322755Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:14:43.0323239Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:14:43.0323743Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:14:43.0324228Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:14:43.0324732Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:14:43.0325209Z parm: NVreg_RmMsg:charp 2024-08-20T22:14:43.0325629Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:14:43.0326101Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:14:43.0326576Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:14:43.0327032Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:14:43.0327610Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:14:43.0328093Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:14:43.0328546Z parm: rm_firmware_active:charp 2024-08-20T22:14:43.0328951Z + HAS_NVIDIA_DRIVER=0 2024-08-20T22:14:43.0329343Z ++ command -v nvidia-smi 2024-08-20T22:14:43.0329748Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-20T22:14:43.0330149Z + set +e 2024-08-20T22:14:43.0330668Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-20T22:14:43.0579028Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-20T22:14:43.0579618Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:14:43.0580129Z + '[' 0 -ne 0 ']' 2024-08-20T22:14:43.0580605Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-20T22:14:43.0580989Z + HAS_NVIDIA_DRIVER=1 2024-08-20T22:14:43.0581717Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-20T22:14:43.0582518Z + set -e 2024-08-20T22:14:43.0583074Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-20T22:14:43.0583769Z + '[' 1 -eq 0 ']' 2024-08-20T22:14:43.0584110Z + post_install_nvidia_driver_common 2024-08-20T22:14:43.0586452Z + sudo modprobe nvidia 2024-08-20T22:14:43.2032751Z + echo 'After installing NVIDIA driver' 2024-08-20T22:14:43.2033206Z + lspci 2024-08-20T22:14:43.2033509Z After installing NVIDIA driver 2024-08-20T22:14:43.2138657Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:14:43.2139597Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:14:43.2140539Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:14:43.2141335Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:14:43.2142137Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:14:43.2143060Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:14:43.2143815Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:14:43.2144890Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:14:43.2145479Z + lsmod 2024-08-20T22:14:43.2175963Z Module Size Used by 2024-08-20T22:14:43.2176390Z xt_nat 16384 0 2024-08-20T22:14:43.2176848Z ib_core 454656 0 2024-08-20T22:14:43.2177353Z veth 36864 0 2024-08-20T22:14:43.2177834Z nvidia_modeset 1351680 0 2024-08-20T22:14:43.2178381Z video 65536 1 nvidia_modeset 2024-08-20T22:14:43.2178949Z wmi 36864 1 video 2024-08-20T22:14:43.2179452Z nvidia_uvm 4706304 0 2024-08-20T22:14:43.2179931Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-20T22:14:43.2180422Z drm 602112 1 nvidia 2024-08-20T22:14:43.2180872Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:14:43.2181398Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-20T22:14:43.2181920Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:14:43.2182433Z xt_conntrack 16384 1 2024-08-20T22:14:43.2182811Z nft_chain_nat 16384 3 2024-08-20T22:14:43.2183185Z xt_MASQUERADE 20480 1 2024-08-20T22:14:43.2183656Z nf_nat 57344 3 xt_nat,nft_chain_nat,xt_MASQUERADE 2024-08-20T22:14:43.2184163Z nf_conntrack_netlink 57344 0 2024-08-20T22:14:43.2184776Z nf_conntrack 184320 5 xt_conntrack,nf_nat,xt_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:14:43.2185434Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:14:43.2185896Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:14:43.2186328Z xfrm_user 57344 1 2024-08-20T22:14:43.2186728Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:14:43.2187139Z xt_addrtype 16384 2 2024-08-20T22:14:43.2187518Z nft_compat 20480 4 2024-08-20T22:14:43.2187970Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:14:43.2188806Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:14:43.2189358Z br_netfilter 36864 0 2024-08-20T22:14:43.2189769Z bridge 307200 1 br_netfilter 2024-08-20T22:14:43.2190211Z stp 16384 1 bridge 2024-08-20T22:14:43.2190621Z llc 16384 2 bridge,stp 2024-08-20T22:14:43.2191044Z overlay 167936 0 2024-08-20T22:14:43.2191414Z tls 114688 0 2024-08-20T22:14:43.2191784Z nls_ascii 16384 1 2024-08-20T22:14:43.2192149Z nls_cp437 20480 1 2024-08-20T22:14:43.2192519Z vfat 24576 1 2024-08-20T22:14:43.2192886Z sunrpc 692224 1 2024-08-20T22:14:43.2193259Z fat 86016 1 vfat 2024-08-20T22:14:43.2193650Z ena 167936 0 2024-08-20T22:14:43.2194023Z ghash_clmulni_intel 16384 0 2024-08-20T22:14:43.2194406Z ptp 36864 1 ena 2024-08-20T22:14:43.2194809Z pps_core 24576 1 ptp 2024-08-20T22:14:43.2195209Z aesni_intel 393216 0 2024-08-20T22:14:43.2195584Z i8042 45056 0 2024-08-20T22:14:43.2195958Z serio 28672 3 i8042 2024-08-20T22:14:43.2196380Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:14:43.2196925Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:14:43.2197575Z button 24576 0 2024-08-20T22:14:43.2198068Z sch_fq_codel 20480 17 2024-08-20T22:14:43.2198552Z dm_mod 188416 0 2024-08-20T22:14:43.2199026Z fuse 163840 1 2024-08-20T22:14:43.2199475Z configfs 57344 1 2024-08-20T22:14:43.2199858Z dax 45056 1 dm_mod 2024-08-20T22:14:43.2200259Z loop 36864 0 2024-08-20T22:14:43.2200630Z dmi_sysfs 20480 0 2024-08-20T22:14:43.2201005Z crc32_pclmul 16384 0 2024-08-20T22:14:43.2201371Z crc32c_intel 24576 0 2024-08-20T22:14:43.2201754Z efivarfs 24576 1 2024-08-20T22:14:43.2202277Z + modinfo nvidia 2024-08-20T22:14:43.2202987Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:14:43.2203684Z alias: char-major-195-* 2024-08-20T22:14:43.2204072Z version: 550.54.15 2024-08-20T22:14:43.2204428Z supported: external 2024-08-20T22:14:43.2204776Z license: NVIDIA 2024-08-20T22:14:43.2205160Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:14:43.2205650Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:14:43.2206115Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:14:43.2206586Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:14:43.2207081Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:14:43.2207582Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:14:43.2208081Z depends: i2c-core,drm 2024-08-20T22:14:43.2208450Z retpoline: Y 2024-08-20T22:14:43.2208766Z name: nvidia 2024-08-20T22:14:43.2209382Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:14:43.2210098Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:14:43.2210749Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:14:43.2211345Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:14:43.2211800Z parm: NVreg_RmLogonRC:int 2024-08-20T22:14:43.2212237Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:14:43.2212694Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:14:43.2213136Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:14:43.2213582Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:14:43.2214116Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:14:43.2214678Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:14:43.2215163Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:14:43.2215700Z parm: NVreg_EnableMSI:int 2024-08-20T22:14:43.2216129Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:14:43.2216592Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:14:43.2217130Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:14:43.2217714Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:14:43.2218279Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:14:43.2218890Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:43.2219492Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:14:43.2220138Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:43.2220753Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:14:43.2221247Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:14:43.2221781Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:14:43.2222420Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:14:43.2222928Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:14:43.2223395Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:14:43.2223882Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:14:43.2224361Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:14:43.2224806Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:14:43.2225321Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:14:43.2225855Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:14:43.2226338Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:14:43.2226821Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:14:43.2227321Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:14:43.2227812Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:14:43.2228318Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:14:43.2228794Z parm: NVreg_RmMsg:charp 2024-08-20T22:14:43.2229210Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:14:43.2229685Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:14:43.2230254Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:14:43.2230715Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:14:43.2231198Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:14:43.2231679Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:14:43.2232139Z parm: rm_firmware_active:charp 2024-08-20T22:14:43.2232545Z + set +e 2024-08-20T22:14:43.2232869Z + nvidia-smi 2024-08-20T22:14:43.2407117Z Tue Aug 20 22:14:43 2024 2024-08-20T22:14:43.2407892Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:43.2408779Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:14:43.2409566Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:14:43.2410432Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:14:43.2411329Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:14:43.2412030Z | | | MIG M. | 2024-08-20T22:14:43.2412585Z |=========================================+========================+======================| 2024-08-20T22:14:43.2647445Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:14:43.2648177Z | 0% 28C P8 20W / 300W | 0MiB / 23028MiB | 0% Default | 2024-08-20T22:14:43.2648811Z | | | N/A | 2024-08-20T22:14:43.2649515Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:14:43.2652675Z 2024-08-20T22:14:43.2653560Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:43.2654213Z | Processes: | 2024-08-20T22:14:43.2654915Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:14:43.2655601Z | ID ID Usage | 2024-08-20T22:14:43.2656167Z |=========================================================================================| 2024-08-20T22:14:43.2658210Z | No running processes found | 2024-08-20T22:14:43.2659003Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:43.5431474Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-20T22:14:43.5623356Z NVIDIA A10G 2024-08-20T22:14:43.5688043Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:14:43.5688467Z + '[' 0 -eq 0 ']' 2024-08-20T22:14:43.5689198Z + echo 'INFO: Ignoring allowed status 0' 2024-08-20T22:14:43.5689670Z + set -e 2024-08-20T22:14:43.5690220Z INFO: Ignoring allowed status 0 2024-08-20T22:14:43.5697502Z == Installing nvidia container toolkit for amzn2023 == 2024-08-20T22:14:43.5701175Z + sudo yum install -y yum-utils 2024-08-20T22:14:44.1110989Z Last metadata expiration check: 0:34:10 ago on Tue Aug 20 21:40:34 2024. 2024-08-20T22:14:44.1332021Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-20T22:14:44.1632835Z Dependencies resolved. 2024-08-20T22:14:44.1756451Z Nothing to do. 2024-08-20T22:14:44.1756820Z Complete! 2024-08-20T22:14:44.2458034Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-20T22:14:44.2459464Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:14:44.2460840Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:14:44.5115863Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:14:44.5809662Z + sudo yum install -y nvidia-docker2 2024-08-20T22:14:45.1034458Z nvidia-container-toolkit 11 kB/s | 833 B 00:00 2024-08-20T22:14:45.1254164Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-20T22:14:45.1554932Z Dependencies resolved. 2024-08-20T22:14:45.1678101Z Nothing to do. 2024-08-20T22:14:45.1678438Z Complete! 2024-08-20T22:14:45.2621094Z + sudo systemctl restart docker 2024-08-20T22:15:14.8707247Z nvidia-persistenced failed to initialize. Check syslog for more details. 2024-08-20T22:15:14.8937531Z Tue Aug 20 22:15:14 2024 2024-08-20T22:15:14.8939987Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:15:14.8940867Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:15:14.8941751Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:15:14.8942710Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:15:14.8943620Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:15:14.8944297Z | | | MIG M. | 2024-08-20T22:15:14.8944846Z |=========================================+========================+======================| 2024-08-20T22:15:14.9139579Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:15:14.9140322Z | 0% 28C P8 20W / 300W | 0MiB / 23028MiB | 0% Default | 2024-08-20T22:15:14.9140961Z | | | N/A | 2024-08-20T22:15:14.9141941Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:15:14.9144709Z 2024-08-20T22:15:14.9145420Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:15:14.9146055Z | Processes: | 2024-08-20T22:15:14.9146764Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:15:14.9147458Z | ID ID Usage | 2024-08-20T22:15:14.9148018Z |=========================================================================================| 2024-08-20T22:15:14.9151296Z | No running processes found | 2024-08-20T22:15:14.9152120Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:15:15.8626003Z Command completed after 1 attempt(s). 2024-08-20T22:15:15.8709538Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:15:15.8710331Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:15:15.8711022Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T22:15:15.8711725Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:15.8725999Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:15.8726506Z env: 2024-08-20T22:15:15.8726790Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:15.8727243Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:15.8727739Z ##[endgroup] 2024-08-20T22:15:16.1224211Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:15:16.1397856Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-08-20T22:15:16.1402878Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-08-20T22:15:16.2630268Z Prepare all required actions 2024-08-20T22:15:16.2630792Z Getting action download info 2024-08-20T22:15:16.3713702Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T22:15:16.5625543Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T22:15:16.7082619Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T22:15:16.7083108Z with: 2024-08-20T22:15:16.7083466Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:16.7083933Z s3-bucket: gha-artifacts 2024-08-20T22:15:16.7084278Z env: 2024-08-20T22:15:16.7084560Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:16.7085022Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:16.7085516Z ##[endgroup] 2024-08-20T22:15:16.7120745Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:15:16.7121204Z with: 2024-08-20T22:15:16.7121642Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:16.7122110Z s3-bucket: gha-artifacts 2024-08-20T22:15:16.7122468Z region: us-east-1 2024-08-20T22:15:16.7122784Z env: 2024-08-20T22:15:16.7123060Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:16.7123521Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:16.7124015Z ##[endgroup] 2024-08-20T22:15:17.1950824Z (node:291834) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:15:17.1951583Z 2024-08-20T22:15:17.1951857Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:15:17.1952635Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:15:17.1953892Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:15:17.2737001Z Found 1 objects with prefix pytorch/pytorch/10479310721/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-08-20T22:15:17.2738198Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:15:24.9156278Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:15:24.9165723Z Artifact download has finished successfully 2024-08-20T22:15:24.9489201Z ##[group]Run unzip -o artifacts.zip 2024-08-20T22:15:24.9489643Z unzip -o artifacts.zip 2024-08-20T22:15:24.9498613Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:24.9499126Z env: 2024-08-20T22:15:24.9499417Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:24.9499877Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:24.9500369Z ##[endgroup] 2024-08-20T22:15:24.9557522Z Archive: artifacts.zip 2024-08-20T22:15:24.9558979Z creating: dist/ 2024-08-20T22:15:27.0856696Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:15:27.0857371Z creating: build/custom_test_artifacts/ 2024-08-20T22:15:27.0857973Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T22:15:27.0858732Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T22:15:27.0859578Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:27.0865855Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:27.0866792Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:27.0867742Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:27.0868748Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:27.0869728Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:27.0871223Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:27.0873081Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:27.0874419Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:27.0875442Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:27.0876915Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:27.0878489Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:27.0880096Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:27.0881471Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:27.0882897Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:27.0884511Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:27.0885907Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:27.0886931Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:27.0926732Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:27.0966972Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:27.0968711Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:27.1013707Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:27.1016256Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:27.1019776Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:27.1023680Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:27.1026350Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:27.1027889Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:27.1029369Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:27.1030769Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:27.1032347Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:27.1033655Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:27.1034911Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:27.1036135Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:27.1037698Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:27.1038910Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:27.1040121Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:27.1089673Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:27.1150161Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:27.1151557Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:27.1152756Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:27.1153826Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:27.1154952Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:27.1156125Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T22:15:27.1157339Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T22:15:27.1158704Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T22:15:27.1160088Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T22:15:27.1161228Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T22:15:27.1162302Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T22:15:27.1163385Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T22:15:27.1164476Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T22:15:27.1165605Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T22:15:27.1166666Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T22:15:27.1182164Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T22:15:27.1327116Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T22:15:27.1328337Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T22:15:27.1329655Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T22:15:27.1331126Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T22:15:27.1332442Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T22:15:27.1333714Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T22:15:27.1335070Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T22:15:27.1336276Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T22:15:27.1337663Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T22:15:27.1338791Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T22:15:27.1339912Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T22:15:27.1357022Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T22:15:27.1440742Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T22:15:27.1442287Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:27.1443580Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:27.1445079Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T22:15:27.1446432Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T22:15:27.1447345Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:27.1448262Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-20T22:15:27.1449310Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T22:15:27.1450265Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T22:15:27.1451401Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T22:15:27.1573506Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T22:15:27.1635742Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T22:15:27.1636861Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T22:15:27.1637616Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T22:15:27.1638439Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:27.1644884Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:27.1645967Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:27.1647144Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:27.1648160Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:27.1649296Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:27.1650494Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:27.1651927Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:27.1653019Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:27.1654016Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:27.1655455Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:27.1657079Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:27.1658622Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:27.1659950Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:27.1661390Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:27.1663051Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:27.1664177Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:27.1665241Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:27.1705535Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:27.1745377Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:27.1746817Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:27.1792412Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:27.1794358Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:27.1796670Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:27.1798438Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:27.1800009Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:27.1801664Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:27.1803280Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:27.1804868Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:27.1806399Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:27.1807763Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:27.1809191Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:27.1810572Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:27.1811887Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:27.1813069Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:27.1814270Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:27.1869007Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:27.1929309Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:27.1930626Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:27.1932000Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:27.1933073Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:27.1934165Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:27.1935448Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T22:15:27.1936999Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T22:15:27.1938480Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T22:15:27.1939890Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T22:15:27.1941111Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T22:15:27.1942220Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T22:15:27.1943467Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T22:15:27.1944573Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T22:15:27.1945724Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T22:15:27.1946809Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T22:15:27.1962290Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T22:15:27.2026971Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T22:15:27.2028440Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:27.2029691Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:27.2030772Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T22:15:27.2031879Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T22:15:27.2032816Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:27.2033719Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-20T22:15:27.2034944Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T22:15:27.2036080Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T22:15:27.2037297Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T22:15:27.2087764Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T22:15:27.2088596Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T22:15:27.2089456Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T22:15:27.2090337Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:27.2097054Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:27.2098196Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:27.2099281Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:27.2100339Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:27.2101549Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:27.2102913Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:27.2104240Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:27.2105306Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:27.2106556Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:27.2107833Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:27.2109239Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:27.2110832Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:27.2112179Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:27.2113580Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:27.2115009Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:27.2116237Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:27.2117310Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:27.2157297Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:27.2197314Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:27.2199152Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:27.2244457Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:27.2246428Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:27.2248202Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:27.2250077Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:27.2251779Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:27.2253472Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:27.2255146Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:27.2256887Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:27.2258439Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:27.2259910Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:27.2261207Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:27.2262594Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:27.2263908Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:27.2265174Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:27.2266454Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:27.2319514Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:27.2380085Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:27.2381535Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:27.2382914Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:27.2384033Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:27.2385223Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:27.2386496Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T22:15:27.2387803Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T22:15:27.2389304Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T22:15:27.2390718Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T22:15:27.2391945Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T22:15:27.2393139Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T22:15:27.2394523Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T22:15:27.2396143Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T22:15:27.2397350Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T22:15:27.2398526Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T22:15:27.2399802Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T22:15:27.2522252Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T22:15:27.2523648Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T22:15:27.2525086Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T22:15:27.2526567Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T22:15:27.2528042Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T22:15:27.2529489Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T22:15:27.2530804Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T22:15:27.2532053Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T22:15:27.2533273Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T22:15:27.2534513Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T22:15:27.2535790Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T22:15:27.2552235Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T22:15:27.2607605Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T22:15:27.2609209Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:27.2610579Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:27.2611798Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T22:15:27.2612910Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T22:15:27.2614031Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:27.2615022Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-20T22:15:27.2616656Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T22:15:27.2617926Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T22:15:27.2619111Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T22:15:27.2722965Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T22:15:27.2767269Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T22:15:27.2768542Z creating: build/lib/ 2024-08-20T22:15:27.2857196Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T22:15:27.3313565Z inflating: build/lib/libprotobuf.a 2024-08-20T22:15:27.3323279Z inflating: build/lib/libpthreadpool.a 2024-08-20T22:15:27.3331580Z inflating: build/lib/libcpuinfo.a 2024-08-20T22:15:27.3341014Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T22:15:27.3341633Z inflating: build/lib/libclog.a 2024-08-20T22:15:27.3360511Z inflating: build/lib/libnnpack.a 2024-08-20T22:15:27.3363439Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T22:15:27.3429004Z inflating: build/lib/libgtest.a 2024-08-20T22:15:27.3504644Z inflating: build/lib/libbenchmark.a 2024-08-20T22:15:27.3569475Z inflating: build/lib/libasmjit.a 2024-08-20T22:15:27.3576354Z inflating: build/lib/libittnotify.a 2024-08-20T22:15:27.3604270Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T22:15:27.3732679Z inflating: build/lib/libgloo.a 2024-08-20T22:15:27.3753468Z inflating: build/lib/libfmt.a 2024-08-20T22:15:27.3848683Z inflating: build/lib/libc10.so 2024-08-20T22:15:27.3850743Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-20T22:15:27.3851718Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T22:15:27.4357650Z inflating: build/lib/libprotoc.a 2024-08-20T22:15:27.4377262Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T22:15:27.4395538Z inflating: build/lib/libgmock.a 2024-08-20T22:15:27.4396176Z inflating: build/lib/libgtest_main.a 2024-08-20T22:15:27.4397076Z inflating: build/lib/libbenchmark_main.a 2024-08-20T22:15:27.4788194Z inflating: build/lib/libgloo_cuda.a 2024-08-20T22:15:28.5048265Z inflating: build/lib/libdnnl.a 2024-08-20T22:15:28.5624639Z inflating: build/lib/libtensorpipe.a 2024-08-20T22:15:28.5684339Z inflating: build/lib/libc10_cuda.so 2024-08-20T22:15:28.5685032Z inflating: build/lib/libgmock_main.a 2024-08-20T22:15:28.6953384Z inflating: build/lib/libfbgemm.a 2024-08-20T22:15:28.7209995Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-20T22:15:28.7717636Z inflating: build/lib/libkineto.a 2024-08-20T22:15:28.7902166Z inflating: build/lib/libXNNPACK.a 2024-08-20T22:15:28.7945119Z inflating: build/lib/libonnx_proto.a 2024-08-20T22:15:28.8642228Z inflating: build/lib/libonnx.a 2024-08-20T22:15:31.3159855Z inflating: build/lib/libtorch_cpu.so 2024-08-20T22:15:31.3164722Z inflating: build/lib/libunbox_lib.a 2024-08-20T22:15:31.3169082Z inflating: build/lib/libshm.so 2024-08-20T22:15:33.3851500Z inflating: build/lib/libtorch_cuda.so 2024-08-20T22:15:33.3852446Z inflating: build/lib/libtorch.so 2024-08-20T22:15:33.3855374Z inflating: build/lib/libc10d_cuda_test.so 2024-08-20T22:15:34.2054413Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-20T22:15:34.4057077Z inflating: build/lib/libtorch_python.so 2024-08-20T22:15:34.4129349Z inflating: build/lib/libtorchbind_test.so 2024-08-20T22:15:34.4150626Z inflating: build/lib/libjitbackend_test.so 2024-08-20T22:15:34.4176317Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T22:15:34.4202655Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T22:15:34.4237527Z inflating: build/lib/libnnapi_backend.so 2024-08-20T22:15:34.4238067Z creating: build/bin/ 2024-08-20T22:15:34.4288885Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T22:15:34.4340827Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T22:15:34.4392325Z inflating: build/bin/c10_Device_test 2024-08-20T22:15:34.4451984Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T22:15:34.4506011Z inflating: build/bin/c10_Scalar_test 2024-08-20T22:15:34.4555639Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T22:15:34.4607440Z inflating: build/bin/c10_SymInt_test 2024-08-20T22:15:34.4662235Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T22:15:34.4718746Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T22:15:34.4775611Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T22:15:34.4847474Z inflating: build/bin/c10_cow_test 2024-08-20T22:15:34.4900868Z inflating: build/bin/c10_Bitset_test 2024-08-20T22:15:34.4950504Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T22:15:34.5000989Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T22:15:34.5052149Z inflating: build/bin/c10_Half_test 2024-08-20T22:15:34.5109033Z inflating: build/bin/c10_LeftRight_test 2024-08-20T22:15:34.5164782Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T22:15:34.5215266Z inflating: build/bin/c10_Synchronized_test 2024-08-20T22:15:34.5271602Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T22:15:34.5323587Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T22:15:34.5376214Z inflating: build/bin/c10_TypeList_test 2024-08-20T22:15:34.5424891Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T22:15:34.5477175Z inflating: build/bin/c10_accumulate_test 2024-08-20T22:15:34.5532844Z inflating: build/bin/c10_bfloat16_test 2024-08-20T22:15:34.5583867Z inflating: build/bin/c10_bit_cast_test 2024-08-20T22:15:34.5640795Z inflating: build/bin/c10_complex_math_test 2024-08-20T22:15:34.5696763Z inflating: build/bin/c10_complex_test 2024-08-20T22:15:34.5750232Z inflating: build/bin/c10_exception_test 2024-08-20T22:15:34.5801406Z inflating: build/bin/c10_flags_test 2024-08-20T22:15:34.5852802Z inflating: build/bin/c10_generic_math_test 2024-08-20T22:15:34.6018172Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T22:15:34.6069614Z inflating: build/bin/c10_irange_test 2024-08-20T22:15:34.6124410Z inflating: build/bin/c10_lazy_test 2024-08-20T22:15:34.6182069Z inflating: build/bin/c10_logging_test 2024-08-20T22:15:34.6256894Z inflating: build/bin/c10_optional_test 2024-08-20T22:15:34.6319724Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T22:15:34.6373988Z inflating: build/bin/c10_registry_test 2024-08-20T22:15:34.6525804Z inflating: build/bin/c10_small_vector_test 2024-08-20T22:15:34.6578136Z inflating: build/bin/c10_ssize_test 2024-08-20T22:15:34.6630392Z inflating: build/bin/c10_string_util_test 2024-08-20T22:15:34.6682477Z inflating: build/bin/c10_tempfile_test 2024-08-20T22:15:34.6740799Z inflating: build/bin/c10_string_view_test 2024-08-20T22:15:34.6789800Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T22:15:34.6846250Z inflating: build/bin/c10_typeid_test 2024-08-20T22:15:34.7295187Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T22:15:34.7743881Z inflating: build/bin/protoc 2024-08-20T22:15:34.7797281Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-20T22:15:34.7851031Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-20T22:15:34.7904201Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-20T22:15:34.7956974Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-20T22:15:34.8011253Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-20T22:15:34.8063845Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-20T22:15:34.8113090Z inflating: build/bin/c10_cuda_CUDATest 2024-08-20T22:15:34.8166531Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-20T22:15:34.8498716Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T22:15:34.8845644Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T22:15:34.9207986Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T22:15:34.9261472Z inflating: build/bin/BackoffTest 2024-08-20T22:15:34.9314874Z inflating: build/bin/FileStoreTest 2024-08-20T22:15:34.9371274Z inflating: build/bin/TCPStoreTest 2024-08-20T22:15:34.9425129Z inflating: build/bin/HashStoreTest 2024-08-20T22:15:34.9439404Z inflating: build/bin/ProcessGroupMPITest 2024-08-20T22:15:34.9493697Z inflating: build/bin/test_edge_op_registration 2024-08-20T22:15:34.9498122Z inflating: build/bin/torch_shm_manager 2024-08-20T22:15:34.9501011Z inflating: build/bin/example_allreduce 2024-08-20T22:15:34.9557420Z inflating: build/bin/test_dist_autograd 2024-08-20T22:15:34.9626584Z inflating: build/bin/test_cpp_rpc 2024-08-20T22:15:34.9629336Z inflating: build/bin/parallel_benchmark 2024-08-20T22:15:34.9697202Z inflating: build/bin/test_mobile_nnc 2024-08-20T22:15:34.9706567Z inflating: build/bin/aot_model_compiler_test 2024-08-20T22:15:35.0055009Z inflating: build/bin/test_lazy 2024-08-20T22:15:35.1244430Z inflating: build/bin/test_api 2024-08-20T22:15:35.1318979Z inflating: build/bin/Dict_test 2024-08-20T22:15:35.1371795Z inflating: build/bin/Dimname_test 2024-08-20T22:15:35.1436927Z inflating: build/bin/MaybeOwned_test 2024-08-20T22:15:35.1494956Z inflating: build/bin/NamedTensor_test 2024-08-20T22:15:35.1555312Z inflating: build/bin/apply_utils_test 2024-08-20T22:15:35.1614569Z inflating: build/bin/atest 2024-08-20T22:15:35.1678241Z inflating: build/bin/basic 2024-08-20T22:15:35.1733412Z inflating: build/bin/broadcast_test 2024-08-20T22:15:35.1784872Z inflating: build/bin/cpu_allocator_test 2024-08-20T22:15:35.1843694Z inflating: build/bin/cpu_generator_test 2024-08-20T22:15:35.1898231Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T22:15:35.1991970Z inflating: build/bin/cpu_rng_test 2024-08-20T22:15:35.2042630Z inflating: build/bin/dispatch_key_set_test 2024-08-20T22:15:35.2093315Z inflating: build/bin/dlconvertor_test 2024-08-20T22:15:35.2152079Z inflating: build/bin/extension_backend_test 2024-08-20T22:15:35.2207816Z inflating: build/bin/half_test 2024-08-20T22:15:35.2304345Z inflating: build/bin/ivalue_test 2024-08-20T22:15:35.2354521Z inflating: build/bin/lazy_tensor_test 2024-08-20T22:15:35.2410130Z inflating: build/bin/math_kernel_test 2024-08-20T22:15:35.2464544Z inflating: build/bin/memory_format_test 2024-08-20T22:15:35.2517500Z inflating: build/bin/memory_overlapping_test 2024-08-20T22:15:35.2571865Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T22:15:35.2628506Z inflating: build/bin/native_test 2024-08-20T22:15:35.2679661Z inflating: build/bin/operator_name_test 2024-08-20T22:15:35.2732127Z inflating: build/bin/operators_test 2024-08-20T22:15:35.2785242Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T22:15:35.2852677Z inflating: build/bin/pow_test 2024-08-20T22:15:35.2910244Z inflating: build/bin/quantized_test 2024-08-20T22:15:35.2961980Z inflating: build/bin/reduce_ops_test 2024-08-20T22:15:35.3014084Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T22:15:35.3070734Z inflating: build/bin/scalar_tensor_test 2024-08-20T22:15:35.3130478Z inflating: build/bin/scalar_test 2024-08-20T22:15:35.3182962Z inflating: build/bin/StorageUtils_test 2024-08-20T22:15:35.3235176Z inflating: build/bin/stride_properties_test 2024-08-20T22:15:35.3314417Z inflating: build/bin/tensor_iterator_test 2024-08-20T22:15:35.3369903Z inflating: build/bin/test_parallel 2024-08-20T22:15:35.3372984Z inflating: build/bin/thread_init_test 2024-08-20T22:15:35.3428824Z inflating: build/bin/type_ptr_test 2024-08-20T22:15:35.3490252Z inflating: build/bin/type_test 2024-08-20T22:15:35.3542276Z inflating: build/bin/undefined_tensor_test 2024-08-20T22:15:35.3544212Z inflating: build/bin/verify_api_visibility 2024-08-20T22:15:35.3614492Z inflating: build/bin/legacy_vmap_test 2024-08-20T22:15:35.3665860Z inflating: build/bin/weakref_test 2024-08-20T22:15:35.3717387Z inflating: build/bin/wrapdim_test 2024-08-20T22:15:35.3770094Z inflating: build/bin/xla_tensor_test 2024-08-20T22:15:35.3831130Z inflating: build/bin/IListRef_test 2024-08-20T22:15:35.3938159Z inflating: build/bin/List_test 2024-08-20T22:15:35.4005202Z inflating: build/bin/KernelFunction_test 2024-08-20T22:15:35.4127970Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T22:15:35.4225629Z inflating: build/bin/kernel_function_test 2024-08-20T22:15:35.4355277Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T22:15:35.4459588Z inflating: build/bin/kernel_lambda_test 2024-08-20T22:15:35.4521157Z inflating: build/bin/kernel_stackbased_test 2024-08-20T22:15:35.4622682Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T22:15:35.4674073Z inflating: build/bin/CppSignature_test 2024-08-20T22:15:35.4723172Z inflating: build/bin/op_allowlist_test 2024-08-20T22:15:35.4780204Z inflating: build/bin/backend_fallback_test 2024-08-20T22:15:35.5084732Z inflating: build/bin/op_registration_test 2024-08-20T22:15:35.5148461Z inflating: build/bin/inline_container_test 2024-08-20T22:15:35.5201371Z inflating: build/bin/cuda_apply_test 2024-08-20T22:15:35.5254038Z inflating: build/bin/cuda_allocator_test 2024-08-20T22:15:35.5313663Z inflating: build/bin/cuda_atomic_ops_test 2024-08-20T22:15:35.5368828Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-20T22:15:35.5440211Z inflating: build/bin/cuda_complex_math_test 2024-08-20T22:15:35.5499999Z inflating: build/bin/cuda_complex_test 2024-08-20T22:15:35.5551138Z inflating: build/bin/cuda_device_test 2024-08-20T22:15:35.5610448Z inflating: build/bin/cuda_cub_test 2024-08-20T22:15:35.5661174Z inflating: build/bin/cuda_dlconvertor_test 2024-08-20T22:15:35.5727353Z inflating: build/bin/cuda_distributions_test 2024-08-20T22:15:35.5784723Z inflating: build/bin/cuda_generator_test 2024-08-20T22:15:35.5834677Z inflating: build/bin/cuda_half_test 2024-08-20T22:15:35.5887414Z inflating: build/bin/cuda_integer_divider_test 2024-08-20T22:15:35.5936829Z inflating: build/bin/cuda_optional_test 2024-08-20T22:15:35.5988968Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-20T22:15:35.6042287Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-20T22:15:35.6093509Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-20T22:15:35.6154691Z inflating: build/bin/cuda_stream_test 2024-08-20T22:15:35.6204233Z inflating: build/bin/cuda_cudnn_test 2024-08-20T22:15:35.6256942Z inflating: build/bin/cuda_vectorized_test 2024-08-20T22:15:35.6271589Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T22:15:35.6337867Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T22:15:35.6395690Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-20T22:15:35.6459804Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-20T22:15:35.6522516Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-20T22:15:35.7355170Z inflating: build/bin/test_tensorexpr 2024-08-20T22:15:35.7940834Z inflating: build/bin/test_jit 2024-08-20T22:15:35.7941929Z creating: .additional_ci_files/ 2024-08-20T22:15:35.8001669Z inflating: .additional_ci_files/test-times.json 2024-08-20T22:15:35.8243722Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T22:15:35.8353533Z ##[group]Run rm artifacts.zip 2024-08-20T22:15:35.8353926Z rm artifacts.zip 2024-08-20T22:15:35.8363098Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:35.8363619Z env: 2024-08-20T22:15:35.8363907Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:35.8364370Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:35.8364853Z ##[endgroup] 2024-08-20T22:15:35.9457206Z ##[group]Run df -H 2024-08-20T22:15:35.9457533Z df -H 2024-08-20T22:15:35.9466632Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:35.9467139Z env: 2024-08-20T22:15:35.9467437Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:35.9467908Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:35.9468443Z ##[endgroup] 2024-08-20T22:15:35.9696258Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T22:15:35.9696955Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T22:15:35.9697546Z tmpfs 34G 0 34G 0% /dev/shm 2024-08-20T22:15:35.9698080Z tmpfs 14G 553k 14G 1% /run 2024-08-20T22:15:35.9698531Z /dev/nvme0n1p1 161G 82G 80G 51% / 2024-08-20T22:15:35.9700057Z tmpfs 34G 967k 34G 1% /tmp 2024-08-20T22:15:35.9701047Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T22:15:35.9701723Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-08-20T22:15:35.9740997Z Prepare all required actions 2024-08-20T22:15:35.9741442Z Getting action download info 2024-08-20T22:15:36.0814375Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T22:15:36.0814845Z with: 2024-08-20T22:15:36.0815109Z env: 2024-08-20T22:15:36.0815390Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:36.0815842Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:36.0816332Z ##[endgroup] 2024-08-20T22:15:36.0987433Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:15:36.0987886Z with: 2024-08-20T22:15:36.0988165Z name: td_results 2024-08-20T22:15:36.0988485Z s3-bucket: gha-artifacts 2024-08-20T22:15:36.0988844Z region: us-east-1 2024-08-20T22:15:36.0989149Z env: 2024-08-20T22:15:36.0989444Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:36.0989895Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:36.0990386Z ##[endgroup] 2024-08-20T22:15:36.5855676Z (node:291856) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:15:36.5856367Z 2024-08-20T22:15:36.5856707Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:15:36.5857564Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:15:36.5858478Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:15:36.6720811Z Found 0 objects with prefix pytorch/pytorch/10479310721/td_results/ 2024-08-20T22:15:36.6727889Z Artifact download has finished successfully 2024-08-20T22:15:36.7062864Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T22:15:36.7063356Z mkdir -p .additional_ci_files 2024-08-20T22:15:36.7063923Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T22:15:36.7072927Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:36.7073431Z env: 2024-08-20T22:15:36.7073724Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:36.7074211Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:36.7074715Z ##[endgroup] 2024-08-20T22:15:36.7454677Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T22:15:36.7471803Z ##[error]Process completed with exit code 1. 2024-08-20T22:15:36.7547968Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T22:15:36.7548467Z .github/scripts/parse_ref.py 2024-08-20T22:15:36.7557321Z shell: /usr/bin/bash -e {0} 2024-08-20T22:15:36.7557678Z env: 2024-08-20T22:15:36.7557960Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:36.7558421Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:36.7558903Z ##[endgroup] 2024-08-20T22:15:36.7914932Z Prepare all required actions 2024-08-20T22:15:36.7953766Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T22:15:36.7954240Z with: 2024-08-20T22:15:36.7954792Z github-token: *** 2024-08-20T22:15:36.7955107Z env: 2024-08-20T22:15:36.7955383Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:36.7955843Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:36.7956338Z ##[endgroup] 2024-08-20T22:15:36.8089064Z ##[group]Run set -eux 2024-08-20T22:15:36.8089413Z set -eux 2024-08-20T22:15:36.8090000Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T22:15:36.8098700Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:36.8099212Z env: 2024-08-20T22:15:36.8099505Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:36.8099970Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:36.8100618Z GITHUB_TOKEN: *** 2024-08-20T22:15:36.8100929Z ##[endgroup] 2024-08-20T22:15:36.8129887Z + python3 .github/scripts/get_workflow_job_id.py 10479310721 i-00e21703921bea6d0 2024-08-20T22:15:37.3954642Z setting job-id=29026297876 2024-08-20T22:15:37.3955719Z setting job-name=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:37.4174011Z Prepare all required actions 2024-08-20T22:15:37.4174469Z Getting action download info 2024-08-20T22:15:37.5379506Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T22:15:37.5379971Z with: 2024-08-20T22:15:37.5380439Z github-token: *** 2024-08-20T22:15:37.5387026Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:15:37.5394371Z job-name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:37.5395273Z env: 2024-08-20T22:15:37.5395551Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:37.5396004Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:37.5396486Z ##[endgroup] 2024-08-20T22:15:37.5652793Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:15:37.5653358Z with: 2024-08-20T22:15:37.5653635Z shell: bash 2024-08-20T22:15:37.5653943Z timeout_minutes: 10 2024-08-20T22:15:37.5654273Z max_attempts: 5 2024-08-20T22:15:37.5654594Z retry_wait_seconds: 30 2024-08-20T22:15:37.5655704Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:15:37.5656871Z polling_interval_seconds: 1 2024-08-20T22:15:37.5657254Z warning_on_retry: true 2024-08-20T22:15:37.5657609Z continue_on_error: false 2024-08-20T22:15:37.5657958Z env: 2024-08-20T22:15:37.5658243Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:37.5658693Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:37.5659341Z GITHUB_TOKEN: *** 2024-08-20T22:15:37.5659668Z ##[endgroup] 2024-08-20T22:15:37.8090023Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:15:38.3472533Z Collecting requests==2.27.1 2024-08-20T22:15:38.3485589Z Using cached requests-2.27.1-py2.py3-none-any.whl.metadata (5.0 kB) 2024-08-20T22:15:38.4032681Z Collecting pyyaml==6.0.1 2024-08-20T22:15:38.4061268Z Using cached PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB) 2024-08-20T22:15:38.4557525Z Collecting urllib3<1.27,>=1.21.1 (from requests==2.27.1) 2024-08-20T22:15:38.4574642Z Using cached urllib3-1.26.19-py2.py3-none-any.whl.metadata (49 kB) 2024-08-20T22:15:38.4646750Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/miniconda/lib/python3.12/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-20T22:15:38.5507196Z Collecting charset-normalizer~=2.0.0 (from requests==2.27.1) 2024-08-20T22:15:38.5600380Z Using cached charset_normalizer-2.0.12-py3-none-any.whl.metadata (11 kB) 2024-08-20T22:15:38.5694277Z Requirement already satisfied: idna<4,>=2.5 in /home/ec2-user/miniconda/lib/python3.12/site-packages (from requests==2.27.1) (3.7) 2024-08-20T22:15:38.5920947Z Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-20T22:15:38.6079548Z Using cached PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (724 kB) 2024-08-20T22:15:38.6691329Z Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-20T22:15:38.6853935Z Using cached urllib3-1.26.19-py2.py3-none-any.whl (143 kB) 2024-08-20T22:15:38.9549326Z Installing collected packages: urllib3, pyyaml, charset-normalizer, requests 2024-08-20T22:15:38.9550276Z Attempting uninstall: urllib3 2024-08-20T22:15:38.9570350Z Found existing installation: urllib3 2.2.2 2024-08-20T22:15:38.9641318Z Uninstalling urllib3-2.2.2: 2024-08-20T22:15:38.9650302Z Successfully uninstalled urllib3-2.2.2 2024-08-20T22:15:39.1732989Z Attempting uninstall: charset-normalizer 2024-08-20T22:15:39.1767811Z Found existing installation: charset-normalizer 3.3.2 2024-08-20T22:15:39.1802945Z Uninstalling charset-normalizer-3.3.2: 2024-08-20T22:15:39.1952432Z Successfully uninstalled charset-normalizer-3.3.2 2024-08-20T22:15:39.2860749Z Attempting uninstall: requests 2024-08-20T22:15:39.2886772Z Found existing installation: requests 2.32.3 2024-08-20T22:15:39.2966932Z Uninstalling requests-2.32.3: 2024-08-20T22:15:39.2973246Z Successfully uninstalled requests-2.32.3 2024-08-20T22:15:39.3882594Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:15:39.3884021Z conda 24.7.1 requires requests<3,>=2.28.0, but you have requests 2.27.1 which is incompatible. 2024-08-20T22:15:39.3885091Z Successfully installed charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 urllib3-1.26.19 2024-08-20T22:15:39.6214427Z Command completed after 1 attempt(s). 2024-08-20T22:15:39.6290013Z ##[group]Run set -x 2024-08-20T22:15:39.6290378Z set -x 2024-08-20T22:15:39.6290696Z  2024-08-20T22:15:39.6291266Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:15:39.6291962Z # in runner workspace 2024-08-20T22:15:39.6292513Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T22:15:39.6302753Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:39.6303261Z env: 2024-08-20T22:15:39.6303545Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:39.6304021Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:39.6304517Z ##[endgroup] 2024-08-20T22:15:39.6337026Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T22:15:39.6594162Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:15:39.6594720Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:15:39.6595194Z echo "Job name: ${JOB_NAME}" 2024-08-20T22:15:39.6595613Z  2024-08-20T22:15:39.6596161Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:15:39.6596846Z # in runner workspace 2024-08-20T22:15:39.6597432Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T22:15:39.6598077Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T22:15:39.6598540Z  --job-name "${JOB_NAME}" \ 2024-08-20T22:15:39.6605435Z  --test-matrix "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-08-20T22:15:39.6612588Z  --selected-test-configs "" \ 2024-08-20T22:15:39.6613059Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T22:15:39.6613490Z  --tag "${TAG}" \ 2024-08-20T22:15:39.6613898Z  --event-name "${EVENT_NAME}" \ 2024-08-20T22:15:39.6614348Z  --schedule "${SCHEDULE}" \ 2024-08-20T22:15:39.6614788Z  --branch "${HEAD_BRANCH}" 2024-08-20T22:15:39.6623825Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:39.6624346Z env: 2024-08-20T22:15:39.6624629Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:39.6625096Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:39.6625774Z GITHUB_TOKEN: *** 2024-08-20T22:15:39.6626612Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:39.6627518Z PR_NUMBER: 2024-08-20T22:15:39.6627837Z TAG: ciflow/inductor/133712 2024-08-20T22:15:39.6628213Z EVENT_NAME: push 2024-08-20T22:15:39.6628530Z SCHEDULE: 2024-08-20T22:15:39.6628822Z HEAD_BRANCH: 2024-08-20T22:15:39.6629130Z ##[endgroup] 2024-08-20T22:15:39.6658642Z Workflow: inductor-periodic 2024-08-20T22:15:39.6659706Z Job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:39.9186027Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T22:15:40.1087627Z ##[group]Run echo "Filtered matrix:" 2024-08-20T22:15:40.1088102Z echo "Filtered matrix:" 2024-08-20T22:15:40.1094882Z echo "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-08-20T22:15:40.1101895Z  2024-08-20T22:15:40.1102186Z echo 2024-08-20T22:15:40.1102717Z echo "Is the current job unstable? False" 2024-08-20T22:15:40.1103177Z  2024-08-20T22:15:40.1103463Z echo 2024-08-20T22:15:40.1103826Z echo "Is keep-going label set? False" 2024-08-20T22:15:40.1104259Z  2024-08-20T22:15:40.1104538Z echo 2024-08-20T22:15:40.1104854Z echo "Renabled issues? " 2024-08-20T22:15:40.1114608Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:40.1115110Z env: 2024-08-20T22:15:40.1115398Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:40.1115866Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:40.1116357Z ##[endgroup] 2024-08-20T22:15:40.1146360Z Filtered matrix: 2024-08-20T22:15:40.1155126Z {include: [{config: dynamo_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-08-20T22:15:40.1164689Z 2024-08-20T22:15:40.1164865Z Is the current job unstable? False 2024-08-20T22:15:40.1165156Z 2024-08-20T22:15:40.1165498Z Is keep-going label set? False 2024-08-20T22:15:40.1165757Z 2024-08-20T22:15:40.1165893Z Renabled issues? 2024-08-20T22:15:40.1222735Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:40.1223458Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:40.1231836Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:40.1232342Z env: 2024-08-20T22:15:40.1232639Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:40.1233107Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:40.1233613Z JOB_TIMEOUT: 240 2024-08-20T22:15:40.1233932Z ##[endgroup] 2024-08-20T22:15:40.1411355Z ##[group]Run set -x 2024-08-20T22:15:40.1411779Z set -x 2024-08-20T22:15:40.1412086Z  2024-08-20T22:15:40.1412625Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T22:15:40.1413169Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T22:15:40.1413732Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T22:15:40.1414242Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T22:15:40.1414661Z else 2024-08-20T22:15:40.1415004Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:15:40.1415427Z fi 2024-08-20T22:15:40.1415711Z  2024-08-20T22:15:40.1416179Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T22:15:40.1416929Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T22:15:40.1417606Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T22:15:40.1418198Z # shellcheck disable=SC2086,SC2090 2024-08-20T22:15:40.1418668Z container_name=$(docker run \ 2024-08-20T22:15:40.1419095Z  ${GPU_FLAG:-} \ 2024-08-20T22:15:40.1419476Z  -e BUILD_ENVIRONMENT \ 2024-08-20T22:15:40.1419874Z  -e PR_NUMBER \ 2024-08-20T22:15:40.1420230Z  -e GITHUB_ACTIONS \ 2024-08-20T22:15:40.1420621Z  -e GITHUB_REPOSITORY \ 2024-08-20T22:15:40.1421027Z  -e GITHUB_WORKFLOW \ 2024-08-20T22:15:40.1421412Z  -e GITHUB_JOB \ 2024-08-20T22:15:40.1421770Z  -e GITHUB_RUN_ID \ 2024-08-20T22:15:40.1422160Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T22:15:40.1422713Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T22:15:40.1423114Z  -e JOB_ID \ 2024-08-20T22:15:40.1423455Z  -e JOB_NAME \ 2024-08-20T22:15:40.1423806Z  -e BASE_SHA \ 2024-08-20T22:15:40.1424152Z  -e BRANCH \ 2024-08-20T22:15:40.1424483Z  -e SHA1 \ 2024-08-20T22:15:40.1424831Z  -e AWS_DEFAULT_REGION \ 2024-08-20T22:15:40.1425236Z  -e IN_WHEEL_TEST \ 2024-08-20T22:15:40.1425617Z  -e SHARD_NUMBER \ 2024-08-20T22:15:40.1425992Z  -e TEST_CONFIG \ 2024-08-20T22:15:40.1426365Z  -e NUM_TEST_SHARDS \ 2024-08-20T22:15:40.1426760Z  -e REENABLED_ISSUES \ 2024-08-20T22:15:40.1427175Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T22:15:40.1427597Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T22:15:40.1428000Z  -e TEST_SHOWLOCALS \ 2024-08-20T22:15:40.1428400Z  -e NO_TEST_TIMEOUT \ 2024-08-20T22:15:40.1428777Z  -e NO_TD \ 2024-08-20T22:15:40.1429121Z  -e TD_DISTRIBUTED \ 2024-08-20T22:15:40.1429559Z  -e PR_LABELS \ 2024-08-20T22:15:40.1429967Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T22:15:40.1430416Z  -e SCCACHE_BUCKET \ 2024-08-20T22:15:40.1430818Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T22:15:40.1431235Z  -e XLA_CUDA \ 2024-08-20T22:15:40.1431640Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T22:15:40.1432144Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T22:15:40.1432678Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T22:15:40.1433205Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T22:15:40.1433684Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T22:15:40.1434143Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T22:15:40.1434600Z  -e DASHBOARD_TAG \ 2024-08-20T22:15:40.1435088Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T22:15:40.1435645Z  --security-opt seccomp=unconfined \ 2024-08-20T22:15:40.1436114Z  --cap-add=SYS_PTRACE \ 2024-08-20T22:15:40.1436805Z  --ipc=host \ 2024-08-20T22:15:40.1437178Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T22:15:40.1437579Z  --tty \ 2024-08-20T22:15:40.1437895Z  --detach \ 2024-08-20T22:15:40.1438444Z  --name="${container_name}" \ 2024-08-20T22:15:40.1438878Z  --user jenkins \ 2024-08-20T22:15:40.1439372Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T22:15:40.1440101Z  -w /var/lib/jenkins/workspace \ 2024-08-20T22:15:40.1440545Z  "${DOCKER_IMAGE}" 2024-08-20T22:15:40.1440907Z ) 2024-08-20T22:15:40.1441324Z # Propagate download.pytorch.org IP to container 2024-08-20T22:15:40.1442238Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T22:15:40.1443202Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T22:15:40.1444117Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T22:15:40.1452664Z shell: /usr/bin/bash -e {0} 2024-08-20T22:15:40.1453019Z env: 2024-08-20T22:15:40.1453303Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:40.1453774Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:40.1454380Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:40.1454897Z PR_NUMBER: 2024-08-20T22:15:40.1455229Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T22:15:40.1455671Z GITHUB_WORKFLOW: inductor-periodic 2024-08-20T22:15:40.1456076Z GITHUB_JOB: test 2024-08-20T22:15:40.1456405Z GITHUB_RUN_ID: 10479310721 2024-08-20T22:15:40.1456778Z GITHUB_RUN_NUMBER: 54565 2024-08-20T22:15:40.1457130Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T22:15:40.1457469Z JOB_ID: 29026297876 2024-08-20T22:15:40.1458309Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:40.1459215Z BRANCH: 2024-08-20T22:15:40.1459606Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:40.1460134Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:40.1460627Z TEST_CONFIG: dynamo_eager_torchbench 2024-08-20T22:15:40.1461045Z SHARD_NUMBER: 1 2024-08-20T22:15:40.1461351Z NUM_TEST_SHARDS: 2 2024-08-20T22:15:40.1461679Z REENABLED_ISSUES: 2024-08-20T22:15:40.1462027Z CONTINUE_THROUGH_ERROR: False 2024-08-20T22:15:40.1462417Z VERBOSE_TEST_LOGS: False 2024-08-20T22:15:40.1462892Z TEST_SHOWLOCALS: False 2024-08-20T22:15:40.1463239Z NO_TEST_TIMEOUT: False 2024-08-20T22:15:40.1463575Z NO_TD: False 2024-08-20T22:15:40.1463879Z TD_DISTRIBUTED: False 2024-08-20T22:15:40.1464290Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T22:15:40.1464807Z SCCACHE_S3_KEY_PREFIX: inductor-periodic 2024-08-20T22:15:40.1465240Z SHM_SIZE: 2g 2024-08-20T22:15:40.1466326Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:15:40.1467484Z XLA_CUDA: 2024-08-20T22:15:40.1467969Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:15:40.1468586Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T22:15:40.1469028Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T22:15:40.1469479Z DASHBOARD_TAG: 2024-08-20T22:15:40.1470157Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T22:15:40.1470612Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T22:15:40.1470992Z ##[endgroup] 2024-08-20T22:15:40.1498938Z + [[ dynamo_eager_torchbench == \m\u\l\t\i\g\p\u ]] 2024-08-20T22:15:40.1499749Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-08-20T22:15:40.1500275Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:15:40.1508065Z +++ nproc --ignore=2 2024-08-20T22:15:40.1592775Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310721 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:15:44.9519994Z + container_name=b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T22:15:44.9526240Z + grep download.pytorch.org /etc/hosts 2024-08-20T22:15:44.9527722Z + docker exec -i b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T22:15:45.0958996Z + echo DOCKER_CONTAINER_ID=b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T22:15:45.0963132Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:15:45.0965944Z + docker exec -t b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T22:15:45.5005065Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T22:15:45.8240500Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T22:15:45.8242548Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T22:15:45.8244709Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T22:15:45.8247490Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T22:15:45.8250225Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T22:15:45.8255356Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.13.1) 2024-08-20T22:15:45.8271190Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T22:15:45.8284767Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T22:15:45.8301370Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.22.4) 2024-08-20T22:15:45.8688131Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T22:15:46.4446320Z Installing collected packages: torch 2024-08-20T22:15:56.9768996Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:15:56.9770787Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T22:15:56.9771531Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T22:15:57.0582595Z + export TERM=vt100 2024-08-20T22:15:57.0583428Z + TERM=vt100 2024-08-20T22:15:57.0585650Z ++ dirname .ci/pytorch/test.sh 2024-08-20T22:15:57.0601060Z + source .ci/pytorch/common.sh 2024-08-20T22:15:57.0604221Z +++ dirname .ci/pytorch/common.sh 2024-08-20T22:15:57.0613462Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T22:15:57.0615683Z +++ declare -f -t trap_add 2024-08-20T22:15:57.0621233Z ++ set -ex 2024-08-20T22:15:57.0621765Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:15:57.0622429Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T22:15:57.0623174Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-08-20T22:15:57.0625776Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T22:15:57.0662474Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T22:15:57.0663654Z + trap_add cleanup_workspace EXIT 2024-08-20T22:15:57.0664488Z + trap_add_cmd=cleanup_workspace 2024-08-20T22:15:57.0665137Z + shift 2024-08-20T22:15:57.0665470Z + for trap_add_name in "$@" 2024-08-20T22:15:57.0671369Z +++ trap -p EXIT 2024-08-20T22:15:57.0674694Z ++ eval 'extract_trap_cmd ' 2024-08-20T22:15:57.0675096Z +++ extract_trap_cmd 2024-08-20T22:15:57.0675563Z +++ printf '%s\n' '' 2024-08-20T22:15:57.0675969Z ++ printf '%s\n' cleanup_workspace 2024-08-20T22:15:57.0678143Z + trap -- ' 2024-08-20T22:15:57.0678613Z cleanup_workspace' EXIT 2024-08-20T22:15:57.0679096Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T22:15:57.7522299Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T22:15:57.7545878Z + echo 'Environment variables:' 2024-08-20T22:15:57.7546298Z Environment variables: 2024-08-20T22:15:57.7546698Z + env 2024-08-20T22:15:57.7584958Z INSTALLED_DB=yes 2024-08-20T22:15:57.7585529Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:15:57.7586092Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:15:57.7586563Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:15:57.7589137Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:15:57.7589829Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:15:57.7590397Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:15:57.7590922Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:15:57.7591516Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:57.7592103Z HOSTNAME=b5dc86f43b5d 2024-08-20T22:15:57.7592954Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7593773Z GITHUB_ACTION=__self 2024-08-20T22:15:57.7594132Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:15:57.7598118Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:15:57.7602429Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:15:57.7602950Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:15:57.7603312Z GITHUB_RUN_NUMBER=54565 2024-08-20T22:15:57.7603681Z TEST_CONFIG=dynamo_eager_torchbench 2024-08-20T22:15:57.7604100Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:15:57.7604584Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:15:57.7605057Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:15:57.7605505Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:15:57.7606170Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:15:57.7606589Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:15:57.7607098Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:15:57.7607764Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:15:57.7608228Z GITHUB_REF_TYPE=tag 2024-08-20T22:15:57.7608568Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:15:57.7608961Z NCCL_VERSION=2.17.1-1 2024-08-20T22:15:57.7609363Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:57.7609814Z XLA_CUDA= 2024-08-20T22:15:57.7610320Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:15:57.7611018Z *** 2024-08-20T22:15:57.7611317Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:15:57.7611719Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:15:57.7612099Z GITHUB_ACTIONS=true 2024-08-20T22:15:57.7612445Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:57.7612963Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:15:57.7613511Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:15:57.7614005Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:57.7614527Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:15:57.7615017Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:57.7616006Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:15:57.7616788Z UCC_HOME=/usr 2024-08-20T22:15:57.7617130Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:15:57.7617526Z VERBOSE_TEST_LOGS=False 2024-08-20T22:15:57.7617872Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:15:57.7618336Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:15:57.7618836Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:15:57.7619301Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:15:57.7619688Z SHARD_NUMBER=1 2024-08-20T22:15:57.7620002Z GITHUB_REF_PROTECTED=false 2024-08-20T22:15:57.7620354Z HOME=/var/lib/jenkins 2024-08-20T22:15:57.7620716Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:15:57.7621165Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:15:57.7621644Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:15:57.7622201Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-20T22:15:57.7622767Z CUDA_VERSION=12.1.1 2024-08-20T22:15:57.7623209Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:15:57.7623658Z NUM_TEST_SHARDS=2 2024-08-20T22:15:57.7623960Z UCX_HOME=/usr 2024-08-20T22:15:57.7624483Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:15:57.7625565Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7627042Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:57.7628431Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7629552Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:15:57.7630214Z GITHUB_EVENT_NAME=push 2024-08-20T22:15:57.7630559Z DASHBOARD_TAG= 2024-08-20T22:15:57.7630864Z GITHUB_RUN_ID=10479310721 2024-08-20T22:15:57.7631338Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:15:57.7631887Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:15:57.7632886Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7633793Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:15:57.7634221Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:15:57.7634605Z PR_NUMBER= 2024-08-20T22:15:57.7634900Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:15:57.7635242Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:15:57.7635693Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:15:57.7636157Z TERM=vt100 2024-08-20T22:15:57.7636782Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:15:57.7637211Z INSTALLED_VISION=yes 2024-08-20T22:15:57.7637698Z BRANCH= 2024-08-20T22:15:57.7637999Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:15:57.7638404Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:15:57.7638947Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:15:57.7639733Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:15:57.7640489Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:15:57.7640975Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:15:57.7641454Z REENABLED_ISSUES= 2024-08-20T22:15:57.7641769Z SHLVL=1 2024-08-20T22:15:57.7642045Z MAX_JOBS=14 2024-08-20T22:15:57.7642368Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:15:57.7642729Z NVARCH=x86_64 2024-08-20T22:15:57.7643039Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:15:57.7643500Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:57.7644039Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:15:57.7644538Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:15:57.7645221Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:15:57.7645791Z GITHUB_JOB=test 2024-08-20T22:15:57.7646200Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:15:57.7646763Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:15:57.7647282Z NO_TEST_TIMEOUT=False 2024-08-20T22:15:57.7647610Z TD_DISTRIBUTED=False 2024-08-20T22:15:57.7647995Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:15:57.7648436Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:15:57.7648865Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:15:57.7649233Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:15:57.7649588Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:15:57.7649942Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:15:57.7650974Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:57.7652039Z GITHUB_BASE_REF= 2024-08-20T22:15:57.7652426Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:15:57.7652805Z CI=true 2024-08-20T22:15:57.7653123Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:15:57.7653540Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:15:57.7653929Z JOB_ID=29026297876 2024-08-20T22:15:57.7654248Z INSTALLED_PROTOBUF=yes 2024-08-20T22:15:57.7654581Z GITHUB_HEAD_REF= 2024-08-20T22:15:57.7654893Z GITHUB_ACTION_REF= 2024-08-20T22:15:57.7655339Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:15:57.7655800Z TEST_SHOWLOCALS=False 2024-08-20T22:15:57.7656199Z GITHUB_WORKFLOW=inductor-periodic 2024-08-20T22:15:57.7656609Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:15:57.7657523Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7658341Z NO_TD=False 2024-08-20T22:15:57.7658688Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:15:57.7659063Z _=/usr/bin/env 2024-08-20T22:15:57.7659550Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T22:15:57.7818663Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-20T22:15:57.7819593Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-20T22:15:57.7820540Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-20T22:15:57.7821673Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-20T22:15:57.7822474Z + BUILD_DIR=build 2024-08-20T22:15:57.7822978Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T22:15:57.7823389Z + BUILD_BIN_DIR=build/bin 2024-08-20T22:15:57.7823750Z + SHARD_NUMBER=1 2024-08-20T22:15:57.7824066Z + NUM_TEST_SHARDS=2 2024-08-20T22:15:57.7824403Z + export VALGRIND=ON 2024-08-20T22:15:57.7824737Z + VALGRIND=ON 2024-08-20T22:15:57.7825241Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-08-20T22:15:57.7825742Z + [[ 0 == \1 ]] 2024-08-20T22:15:57.7826045Z + [[ False == \1 ]] 2024-08-20T22:15:57.7826857Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-08-20T22:15:57.7827579Z ++ realpath build/custom_test_artifacts 2024-08-20T22:15:57.7849784Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T22:15:57.7850855Z + [[ -n '' ]] 2024-08-20T22:15:57.7851221Z + echo 'Environment variables' 2024-08-20T22:15:57.7851608Z Environment variables 2024-08-20T22:15:57.7851928Z + env 2024-08-20T22:15:57.7858830Z INSTALLED_DB=yes 2024-08-20T22:15:57.7859369Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:15:57.7859864Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:15:57.7860379Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:15:57.7861142Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:15:57.7861724Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:15:57.7862253Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:15:57.7863091Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:15:57.7863840Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:57.7864454Z HOSTNAME=b5dc86f43b5d 2024-08-20T22:15:57.7865412Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7866539Z GITHUB_ACTION=__self 2024-08-20T22:15:57.7866952Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:15:57.7870686Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:15:57.7874532Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:15:57.7875069Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:15:57.7875499Z GITHUB_RUN_NUMBER=54565 2024-08-20T22:15:57.7875870Z TEST_CONFIG=dynamo_eager_torchbench 2024-08-20T22:15:57.7876310Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:15:57.7876808Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:15:57.7877291Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:15:57.7877748Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:15:57.7878173Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:15:57.7878590Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:15:57.7879058Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:15:57.7879564Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:15:57.7880043Z GITHUB_REF_TYPE=tag 2024-08-20T22:15:57.7880389Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:15:57.7880788Z NCCL_VERSION=2.17.1-1 2024-08-20T22:15:57.7881197Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:57.7881656Z XLA_CUDA= 2024-08-20T22:15:57.7882123Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:15:57.7882526Z *** 2024-08-20T22:15:57.7882811Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:15:57.7883212Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:15:57.7883594Z GITHUB_ACTIONS=true 2024-08-20T22:15:57.7883930Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:57.7884438Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:15:57.7884984Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:15:57.7885461Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:57.7885978Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:15:57.7886470Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:57.7887405Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:15:57.7888170Z UCC_HOME=/usr 2024-08-20T22:15:57.7888692Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:15:57.7889112Z VERBOSE_TEST_LOGS=False 2024-08-20T22:15:57.7889463Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:15:57.7890029Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:15:57.7890528Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:15:57.7891008Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:15:57.7891401Z SHARD_NUMBER=1 2024-08-20T22:15:57.7891720Z GITHUB_REF_PROTECTED=false 2024-08-20T22:15:57.7892082Z HOME=/var/lib/jenkins 2024-08-20T22:15:57.7892506Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:15:57.7892953Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:15:57.7893435Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:15:57.7893994Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-20T22:15:57.7894416Z CUDA_VERSION=12.1.1 2024-08-20T22:15:57.7894852Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:15:57.7895293Z NUM_TEST_SHARDS=2 2024-08-20T22:15:57.7895604Z UCX_HOME=/usr 2024-08-20T22:15:57.7896138Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:15:57.7897214Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7898689Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:57.7900110Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7901241Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:15:57.7901911Z GITHUB_EVENT_NAME=push 2024-08-20T22:15:57.7902248Z DASHBOARD_TAG= 2024-08-20T22:15:57.7902680Z GITHUB_RUN_ID=10479310721 2024-08-20T22:15:57.7903167Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:15:57.7903717Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:15:57.7904714Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7905633Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:15:57.7906070Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:15:57.7906465Z PR_NUMBER= 2024-08-20T22:15:57.7906761Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:15:57.7907102Z VALGRIND=ON 2024-08-20T22:15:57.7907419Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:15:57.7907874Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:15:57.7908353Z TERM=vt100 2024-08-20T22:15:57.7908708Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:15:57.7909138Z INSTALLED_VISION=yes 2024-08-20T22:15:57.7909467Z BRANCH= 2024-08-20T22:15:57.7909768Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:15:57.7910189Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:15:57.7910631Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:15:57.7911414Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:15:57.7912183Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:15:57.7912684Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:15:57.7913154Z REENABLED_ISSUES= 2024-08-20T22:15:57.7913474Z SHLVL=1 2024-08-20T22:15:57.7913749Z MAX_JOBS=14 2024-08-20T22:15:57.7914081Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:15:57.7914441Z NVARCH=x86_64 2024-08-20T22:15:57.7914750Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:15:57.7915223Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:57.7915791Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:15:57.7916324Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:15:57.7917001Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:15:57.7917580Z GITHUB_JOB=test 2024-08-20T22:15:57.7917990Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:15:57.7918561Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:15:57.7919082Z NO_TEST_TIMEOUT=False 2024-08-20T22:15:57.7919531Z TD_DISTRIBUTED=False 2024-08-20T22:15:57.7919929Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:15:57.7920384Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:15:57.7920905Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:15:57.7921300Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:15:57.7921762Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:15:57.7922228Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:15:57.7923544Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:57.7924720Z GITHUB_BASE_REF= 2024-08-20T22:15:57.7925146Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:15:57.7925628Z CI=true 2024-08-20T22:15:57.7926104Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:15:57.7926646Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:15:57.7927029Z JOB_ID=29026297876 2024-08-20T22:15:57.7927347Z INSTALLED_PROTOBUF=yes 2024-08-20T22:15:57.7927684Z GITHUB_HEAD_REF= 2024-08-20T22:15:57.7928001Z GITHUB_ACTION_REF= 2024-08-20T22:15:57.7928441Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:15:57.7928908Z TEST_SHOWLOCALS=False 2024-08-20T22:15:57.7929319Z GITHUB_WORKFLOW=inductor-periodic 2024-08-20T22:15:57.7929723Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:15:57.7930633Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_51b27757-fc9c-45b5-9c59-4d7be63eda0c 2024-08-20T22:15:57.7931445Z NO_TD=False 2024-08-20T22:15:57.7931745Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:15:57.7932109Z _=/usr/bin/env 2024-08-20T22:15:57.7932480Z + echo 'Testing pytorch' 2024-08-20T22:15:57.7932831Z Testing pytorch 2024-08-20T22:15:57.7933156Z + export LANG=C.UTF-8 2024-08-20T22:15:57.7933503Z + LANG=C.UTF-8 2024-08-20T22:15:57.7933803Z + PR_NUMBER= 2024-08-20T22:15:57.7934163Z + [[ dynamo_eager_torchbench == \d\e\f\a\u\l\t ]] 2024-08-20T22:15:57.7934700Z + [[ dynamo_eager_torchbench == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T22:15:57.7935236Z + [[ dynamo_eager_torchbench == \s\l\o\w ]] 2024-08-20T22:15:57.7935936Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-08-20T22:15:57.7936876Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-08-20T22:15:57.7937433Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:15:57.7937909Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:15:57.7938376Z + [[ dynamo_eager_torchbench == *crossref* ]] 2024-08-20T22:15:57.7938973Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:15:57.7939611Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-08-20T22:15:57.7940269Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:15:57.7940838Z + pip_install --user ninja==1.10.2 2024-08-20T22:15:57.7941386Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T22:15:58.2217672Z Collecting ninja==1.10.2 2024-08-20T22:15:58.2376907Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T22:15:58.2454197Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T22:15:58.7917610Z Installing collected packages: ninja 2024-08-20T22:15:58.7996151Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T22:15:58.7997468Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T22:15:58.8030731Z Successfully installed ninja-1.10.2 2024-08-20T22:15:58.8816691Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:58.8818881Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:58.8820511Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:15:58.8821029Z + install_tlparse 2024-08-20T22:15:58.8821432Z + pip_install --user tlparse==0.3.25 2024-08-20T22:15:58.8822155Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T22:15:59.2715328Z Collecting tlparse==0.3.25 2024-08-20T22:15:59.2895071Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T22:15:59.2961299Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T22:15:59.8598365Z Installing collected packages: tlparse 2024-08-20T22:15:59.8927483Z Successfully installed tlparse-0.3.25 2024-08-20T22:15:59.9690998Z ++ python -m site --user-base 2024-08-20T22:15:59.9932378Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:15:59.9936145Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-08-20T22:15:59.9937267Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-08-20T22:15:59.9938156Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:15:59.9939141Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-08-20T22:15:59.9940289Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-08-20T22:15:59.9940991Z + cd test 2024-08-20T22:15:59.9941526Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T22:16:02.5124134Z + [[ dynamo_eager_torchbench == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T22:16:02.5124829Z + [[ dynamo_eager_torchbench == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T22:16:02.5126962Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T22:16:02.5127570Z + [[ dynamo_eager_torchbench == *pr_time_benchmarks* ]] 2024-08-20T22:16:02.5128254Z + [[ dynamo_eager_torchbench == *dynamo_eager* ]] 2024-08-20T22:16:02.5129033Z + DYNAMO_BENCHMARK_FLAGS+=(--backend eager) 2024-08-20T22:16:02.5129521Z + [[ dynamo_eager_torchbench == *dynamic* ]] 2024-08-20T22:16:02.5130002Z + [[ dynamo_eager_torchbench == *cpu* ]] 2024-08-20T22:16:02.5130502Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T22:16:02.5160022Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:16:02.5160725Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-08-20T22:16:02.5163804Z + cd test 2024-08-20T22:16:02.5164382Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T22:16:03.9826155Z PyTorch built with: 2024-08-20T22:16:03.9826893Z - GCC 9.4 2024-08-20T22:16:03.9827329Z - C++ Version: 201703 2024-08-20T22:16:03.9828339Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:16:03.9829422Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:16:03.9830130Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:16:03.9830653Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T22:16:03.9831145Z - NNPACK is enabled 2024-08-20T22:16:03.9831546Z - CPU capability usage: AVX2 2024-08-20T22:16:03.9831965Z - CUDA Runtime 12.1 2024-08-20T22:16:03.9832492Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-08-20T22:16:03.9833103Z - CuDNN 90.1 (built against CUDA 12.4) 2024-08-20T22:16:03.9833556Z - Magma 2.6.1 2024-08-20T22:16:03.9841065Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T22:16:03.9847381Z 2024-08-20T22:16:04.3052778Z + cd test 2024-08-20T22:16:04.3053594Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T22:16:05.6354717Z ATen/Parallel: 2024-08-20T22:16:05.6355308Z at::get_num_threads() : 8 2024-08-20T22:16:05.6355886Z at::get_num_interop_threads() : 16 2024-08-20T22:16:05.6356497Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:16:05.6357069Z omp_get_max_threads() : 8 2024-08-20T22:16:05.6358572Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:16:05.6359737Z mkl_get_max_threads() : 8 2024-08-20T22:16:05.6360677Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:16:05.6361607Z std::thread::hardware_concurrency() : 16 2024-08-20T22:16:05.6362223Z Environment variables: 2024-08-20T22:16:05.6362759Z OMP_NUM_THREADS : [not set] 2024-08-20T22:16:05.6363332Z MKL_NUM_THREADS : [not set] 2024-08-20T22:16:05.6363910Z ATen parallel backend: OpenMP 2024-08-20T22:16:05.6364274Z 2024-08-20T22:16:05.9137892Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:16:05.9138680Z + [[ dynamo_eager_torchbench == *backward* ]] 2024-08-20T22:16:05.9139325Z + [[ dynamo_eager_torchbench == *xla* ]] 2024-08-20T22:16:05.9139890Z + [[ dynamo_eager_torchbench == *executorch* ]] 2024-08-20T22:16:05.9140422Z + [[ dynamo_eager_torchbench == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T22:16:05.9141110Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:16:05.9141670Z + [[ dynamo_eager_torchbench == distributed ]] 2024-08-20T22:16:05.9142204Z + [[ dynamo_eager_torchbench == *inductor_distributed* ]] 2024-08-20T22:16:05.9142920Z + [[ dynamo_eager_torchbench == *inductor-halide* ]] 2024-08-20T22:16:05.9143582Z + [[ dynamo_eager_torchbench == *inductor-micro-benchmark* ]] 2024-08-20T22:16:05.9144344Z + [[ dynamo_eager_torchbench == *huggingface* ]] 2024-08-20T22:16:05.9144932Z + [[ dynamo_eager_torchbench == *timm* ]] 2024-08-20T22:16:05.9145414Z + [[ dynamo_eager_torchbench == *torchbench* ]] 2024-08-20T22:16:05.9145886Z + [[ dynamo_eager_torchbench == *cpu* ]] 2024-08-20T22:16:05.9146314Z + install_torchaudio cuda 2024-08-20T22:16:05.9146673Z + local commit 2024-08-20T22:16:05.9146993Z ++ get_pinned_commit audio 2024-08-20T22:16:05.9147371Z ++ cat .github/ci_commit_pins/audio.txt 2024-08-20T22:16:05.9165726Z + commit=b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:05.9166359Z + [[ cuda == \c\u\d\a ]] 2024-08-20T22:16:05.9166903Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-20T22:16:05.9167826Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:05.9169282Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:06.2537320Z Collecting git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:06.2539627Z Cloning https://github.com/pytorch/audio.git (to revision b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862) to /tmp/pip-req-build-u_t6plio 2024-08-20T22:16:06.2571708Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio.git /tmp/pip-req-build-u_t6plio 2024-08-20T22:16:07.0281312Z Running command git rev-parse -q --verify 'sha^b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862' 2024-08-20T22:16:07.0308227Z Running command git fetch -q https://github.com/pytorch/audio.git b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:07.4341729Z Resolved https://github.com/pytorch/audio.git to commit b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:16:07.4343065Z Running command git submodule update --init --recursive -q 2024-08-20T22:16:09.8689462Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:16:09.8716846Z [?25hRequirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchaudio==2.4.0a0+b3f6f51) (2.5.0a0+git40ec5f6) 2024-08-20T22:16:09.8745043Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.13.1) 2024-08-20T22:16:09.8749379Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (4.12.2) 2024-08-20T22:16:09.8751356Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2.8.8) 2024-08-20T22:16:09.8754447Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.1.4) 2024-08-20T22:16:09.8757361Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2024.6.1) 2024-08-20T22:16:09.8762280Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (1.13.1) 2024-08-20T22:16:09.8776209Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchaudio==2.4.0a0+b3f6f51) (1.3.0) 2024-08-20T22:16:09.9247348Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchaudio==2.4.0a0+b3f6f51) (2.1.5) 2024-08-20T22:16:09.9310846Z Building wheels for collected packages: torchaudio 2024-08-20T22:18:45.3718905Z Building wheel for torchaudio (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-08-20T22:18:45.3753788Z [?25h Created wheel for torchaudio: filename=torchaudio-2.4.0a0+b3f6f51-cp310-cp310-linux_x86_64.whl size=2310373 sha256=607e7f2076974b49994a37b7de739e505baf9ba805f59943a1ce5cb11f646b95 2024-08-20T22:18:45.3757060Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/b5/02/94/08dddd0295a3ebec4d4a4d9f226adfd865d48fb8ae4af2cc95 2024-08-20T22:18:45.3783395Z Successfully built torchaudio 2024-08-20T22:18:45.8643020Z Installing collected packages: torchaudio 2024-08-20T22:18:46.0731824Z Successfully installed torchaudio-2.4.0a0+b3f6f51 2024-08-20T22:18:46.2557467Z + install_torchtext 2024-08-20T22:18:46.2557888Z + local data_commit 2024-08-20T22:18:46.2558203Z + local text_commit 2024-08-20T22:18:46.2562289Z ++ get_pinned_commit data 2024-08-20T22:18:46.2562752Z ++ cat .github/ci_commit_pins/data.txt 2024-08-20T22:18:46.2580905Z + data_commit=11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:46.2584999Z ++ get_pinned_commit text 2024-08-20T22:18:46.2587414Z ++ cat .github/ci_commit_pins/text.txt 2024-08-20T22:18:46.2604416Z + text_commit=b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:18:46.2605673Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:46.2607563Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:46.6271535Z Collecting git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:46.6277258Z Cloning https://github.com/pytorch/data.git (to revision 11bb5b847ea8b9e0d9bb82db3304daf383008d3f) to /tmp/pip-req-build-isl5y7jz 2024-08-20T22:18:46.6307749Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/data.git /tmp/pip-req-build-isl5y7jz 2024-08-20T22:18:47.0620645Z Running command git rev-parse -q --verify 'sha^11bb5b847ea8b9e0d9bb82db3304daf383008d3f' 2024-08-20T22:18:47.0647944Z Running command git fetch -q https://github.com/pytorch/data.git 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:47.3209558Z Running command git checkout -q 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:47.4658374Z Resolved https://github.com/pytorch/data.git to commit 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:18:47.4659537Z Running command git submodule update --init --recursive -q 2024-08-20T22:20:07.8593735Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:20:07.8630632Z [?25hRequirement already satisfied: urllib3>=1.25 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (1.26.19) 2024-08-20T22:20:07.8633477Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.32.3) 2024-08-20T22:20:07.8637425Z Requirement already satisfied: torch>2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.5.0a0+git40ec5f6) 2024-08-20T22:20:07.8667268Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.13.1) 2024-08-20T22:20:07.8670791Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (4.12.2) 2024-08-20T22:20:07.8674265Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.8.8) 2024-08-20T22:20:07.8677151Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.1.4) 2024-08-20T22:20:07.8680203Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2024.6.1) 2024-08-20T22:20:07.8685047Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.13.1) 2024-08-20T22:20:07.8699392Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.3.0) 2024-08-20T22:20:07.8759241Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.3.2) 2024-08-20T22:20:07.8762764Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.7) 2024-08-20T22:20:07.8767360Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (2024.7.4) 2024-08-20T22:20:07.9262862Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.1.5) 2024-08-20T22:20:07.9327621Z Building wheels for collected packages: torchdata 2024-08-20T22:20:09.9036880Z Building wheel for torchdata (setup.py) ... [?25l- \ | done 2024-08-20T22:20:09.9045958Z [?25h Created wheel for torchdata: filename=torchdata-0.7.0a0+11bb5b8-py3-none-any.whl size=182970 sha256=b12544c1a1ccc558ef8a9686a8a2dfe2472d649a127ca6940ad22eb4f7113527 2024-08-20T22:20:09.9048209Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/ca/59/a3/c8250bfc8d3d4d639498d4beb2e0f0e70b9a508ac61fde85ce 2024-08-20T22:20:09.9075990Z Successfully built torchdata 2024-08-20T22:20:10.3869687Z Installing collected packages: torchdata 2024-08-20T22:20:10.4843758Z Successfully installed torchdata-0.7.0a0+11bb5b8 2024-08-20T22:20:15.5146704Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:15.5148198Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:15.8531924Z Collecting git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:15.8536787Z Cloning https://github.com/pytorch/text.git (to revision b0ebddc648d279826089db91775375221777a2db) to /tmp/pip-req-build-j5hmppfx 2024-08-20T22:20:15.8568256Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/text.git /tmp/pip-req-build-j5hmppfx 2024-08-20T22:20:16.7047890Z Running command git rev-parse -q --verify 'sha^b0ebddc648d279826089db91775375221777a2db' 2024-08-20T22:20:16.7075627Z Running command git fetch -q https://github.com/pytorch/text.git b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:17.2119956Z Running command git checkout -q b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:17.3762068Z Resolved https://github.com/pytorch/text.git to commit b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:20:17.3763215Z Running command git submodule update --init --recursive -q 2024-08-20T22:20:24.9310709Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:20:24.9345698Z [?25hRequirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (4.66.5) 2024-08-20T22:20:24.9347599Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.32.3) 2024-08-20T22:20:24.9351145Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.5.0a0+git40ec5f6) 2024-08-20T22:20:24.9354117Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (1.22.4) 2024-08-20T22:20:24.9357371Z Requirement already satisfied: torchdata in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (0.7.0a0+11bb5b8) 2024-08-20T22:20:24.9383270Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.3.2) 2024-08-20T22:20:24.9387286Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.7) 2024-08-20T22:20:24.9391309Z Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (1.26.19) 2024-08-20T22:20:24.9394848Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (2024.7.4) 2024-08-20T22:20:24.9421298Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.13.1) 2024-08-20T22:20:24.9425614Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (4.12.2) 2024-08-20T22:20:24.9428660Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2.8.8) 2024-08-20T22:20:24.9432616Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.1.4) 2024-08-20T22:20:24.9435070Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2024.6.1) 2024-08-20T22:20:24.9441673Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (1.13.1) 2024-08-20T22:20:24.9456102Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchtext==0.16.0a0+b0ebddc) (1.3.0) 2024-08-20T22:20:25.0042290Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchtext==0.16.0a0+b0ebddc) (2.1.5) 2024-08-20T22:20:25.0108291Z Building wheels for collected packages: torchtext 2024-08-20T22:21:02.9104099Z Building wheel for torchtext (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done 2024-08-20T22:21:02.9138458Z [?25h Created wheel for torchtext: filename=torchtext-0.16.0a0+b0ebddc-cp310-cp310-linux_x86_64.whl size=2045459 sha256=704dde84562fe2db430ea2a6d2f566b60ae975a4302821a96b04fb60ca4953b4 2024-08-20T22:21:02.9140643Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/e5/3d/d7/faafad098ec9437ecdf4495c0f5e72b817fffbb06063e32cbf 2024-08-20T22:21:02.9164558Z Successfully built torchtext 2024-08-20T22:21:03.4023377Z Installing collected packages: torchtext 2024-08-20T22:21:03.5123587Z Successfully installed torchtext-0.16.0a0+b0ebddc 2024-08-20T22:21:03.6742590Z + install_torchvision 2024-08-20T22:21:03.6743080Z + local orig_preload 2024-08-20T22:21:03.6743497Z + local commit 2024-08-20T22:21:03.6747341Z ++ get_pinned_commit vision 2024-08-20T22:21:03.6747927Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T22:21:03.6766053Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:03.6766530Z + orig_preload= 2024-08-20T22:21:03.6767067Z + '[' -n '' ']' 2024-08-20T22:21:03.6767927Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:03.6769380Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:04.0160560Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:04.0165842Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-3_ydmd44 2024-08-20T22:21:04.0194981Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-3_ydmd44 2024-08-20T22:21:05.4695335Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T22:21:05.4725040Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:06.8082152Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:07.1006668Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:21:09.6041328Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-20T22:21:09.6073822Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2024-08-20T22:21:09.6077659Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T22:21:09.6082144Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T22:21:09.6152537Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T22:21:09.6156615Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T22:21:09.6159432Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T22:21:09.6162191Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T22:21:09.6164965Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T22:21:09.6169537Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-20T22:21:09.6183203Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T22:21:09.6657916Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T22:21:09.6719611Z Building wheels for collected packages: torchvision 2024-08-20T22:22:26.4987024Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done 2024-08-20T22:22:26.5019918Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2116050 sha256=2e579f0ff5bcab96fdfef1d7d5d61584b43978cbffa4003b71007a8ccf428353 2024-08-20T22:22:26.5022425Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-20T22:22:26.5056827Z Successfully built torchvision 2024-08-20T22:22:26.9872779Z Installing collected packages: torchvision 2024-08-20T22:22:27.3968207Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T22:22:27.5426648Z + '[' -n '' ']' 2024-08-20T22:22:27.5427122Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-20T22:22:27.5427621Z + pip_install git+https://github.com/pytorch/ao.git 2024-08-20T22:22:27.5428416Z + pip install --progress-bar off git+https://github.com/pytorch/ao.git 2024-08-20T22:22:27.8793572Z Collecting git+https://github.com/pytorch/ao.git 2024-08-20T22:22:27.8798549Z Cloning https://github.com/pytorch/ao.git to /tmp/pip-req-build-ots5wbra 2024-08-20T22:22:27.8828867Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/ao.git /tmp/pip-req-build-ots5wbra 2024-08-20T22:22:28.4491286Z Resolved https://github.com/pytorch/ao.git to commit 2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T22:22:30.8646391Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:22:30.8681386Z [?25hBuilding wheels for collected packages: torchao 2024-08-20T22:23:49.2243779Z Building wheel for torchao (setup.py) ... [?25l- \ | / - \ | / done 2024-08-20T22:23:49.2262313Z [?25h Created wheel for torchao: filename=torchao-0.4.0+git2c8e3f3-cp310-cp310-linux_x86_64.whl size=928019 sha256=9633d60856ad4e749bed673c7ce53addb5af539f69f3ce13005bb34020ef5254 2024-08-20T22:23:49.2266165Z Stored in directory: /tmp/pip-ephem-wheel-cache-j97o_1f3/wheels/c0/df/d8/b1e2f5f1ea3bf141268a83a667137647b234f04c636e1e7a5e 2024-08-20T22:23:49.2295407Z Successfully built torchao 2024-08-20T22:23:49.7706509Z Installing collected packages: torchao 2024-08-20T22:23:50.0153545Z Successfully installed torchao-0.4.0+git2c8e3f3 2024-08-20T22:23:50.1267184Z + id=0 2024-08-20T22:23:50.1267881Z + pip_install opencv-python==4.8.0.74 2024-08-20T22:23:50.1268992Z + pip install --progress-bar off opencv-python==4.8.0.74 2024-08-20T22:23:50.5889292Z Collecting opencv-python==4.8.0.74 2024-08-20T22:23:50.6065000Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB) 2024-08-20T22:23:50.6158466Z Requirement already satisfied: numpy>=1.21.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opencv-python==4.8.0.74) (1.22.4) 2024-08-20T22:23:50.6219659Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB) 2024-08-20T22:23:51.7299031Z Installing collected packages: opencv-python 2024-08-20T22:23:52.6168765Z Successfully installed opencv-python-4.8.0.74 2024-08-20T22:23:52.7143955Z + [[ dynamo_eager_torchbench == *inductor_torchbench_smoketest_perf* ]] 2024-08-20T22:23:52.7144784Z + [[ dynamo_eager_torchbench == *inductor_torchbench_cpu_smoketest_perf* ]] 2024-08-20T22:23:52.7145508Z + [[ dynamo_eager_torchbench == *torchbench_gcp_smoketest* ]] 2024-08-20T22:23:52.7146586Z + checkout_install_torchbench 2024-08-20T22:23:52.7147014Z + local commit 2024-08-20T22:23:52.7149887Z ++ get_pinned_commit torchbench 2024-08-20T22:23:52.7150339Z ++ cat .github/ci_commit_pins/torchbench.txt 2024-08-20T22:23:52.7170122Z + commit=23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T22:23:52.7170739Z + git clone https://github.com/pytorch/benchmark torchbench 2024-08-20T22:23:52.7189591Z Cloning into 'torchbench'... 2024-08-20T22:23:52.8739455Z remote: Enumerating objects: 31001, done. 2024-08-20T22:23:52.8740303Z remote: Counting objects: 0% (1/1741) 2024-08-20T22:23:52.8742123Z remote: Counting objects: 1% (18/1741) 2024-08-20T22:23:52.8744696Z remote: Counting objects: 2% (35/1741) 2024-08-20T22:23:52.8745238Z remote: Counting objects: 3% (53/1741) 2024-08-20T22:23:52.8745779Z remote: Counting objects: 4% (70/1741) 2024-08-20T22:23:52.8746450Z remote: Counting objects: 5% (88/1741) 2024-08-20T22:23:52.8747017Z remote: Counting objects: 6% (105/1741) 2024-08-20T22:23:52.8747555Z remote: Counting objects: 7% (122/1741) 2024-08-20T22:23:52.8748079Z remote: Counting objects: 8% (140/1741) 2024-08-20T22:23:52.8748618Z remote: Counting objects: 9% (157/1741) 2024-08-20T22:23:52.8755972Z remote: Counting objects: 10% (175/1741) 2024-08-20T22:23:52.8756764Z remote: Counting objects: 11% (192/1741) 2024-08-20T22:23:52.8757463Z remote: Counting objects: 12% (209/1741) 2024-08-20T22:23:52.8758188Z remote: Counting objects: 13% (227/1741) 2024-08-20T22:23:52.8758723Z remote: Counting objects: 14% (244/1741) 2024-08-20T22:23:52.8759245Z remote: Counting objects: 15% (262/1741) 2024-08-20T22:23:52.8759907Z remote: Counting objects: 16% (279/1741) 2024-08-20T22:23:52.8760641Z remote: Counting objects: 17% (296/1741) 2024-08-20T22:23:52.8761248Z remote: Counting objects: 18% (314/1741) 2024-08-20T22:23:52.8761910Z remote: Counting objects: 19% (331/1741) 2024-08-20T22:23:52.8762618Z remote: Counting objects: 20% (349/1741) 2024-08-20T22:23:52.8763306Z remote: Counting objects: 21% (366/1741) 2024-08-20T22:23:52.8763980Z remote: Counting objects: 22% (384/1741) 2024-08-20T22:23:52.8764681Z remote: Counting objects: 23% (401/1741) 2024-08-20T22:23:52.8765349Z remote: Counting objects: 24% (418/1741) 2024-08-20T22:23:52.8765930Z remote: Counting objects: 25% (436/1741) 2024-08-20T22:23:52.8766444Z remote: Counting objects: 26% (453/1741) 2024-08-20T22:23:52.8766959Z remote: Counting objects: 27% (471/1741) 2024-08-20T22:23:52.8767515Z remote: Counting objects: 28% (488/1741) 2024-08-20T22:23:52.8768018Z remote: Counting objects: 29% (505/1741) 2024-08-20T22:23:52.8768529Z remote: Counting objects: 30% (523/1741) 2024-08-20T22:23:52.8769041Z remote: Counting objects: 31% (540/1741) 2024-08-20T22:23:52.8769554Z remote: Counting objects: 32% (558/1741) 2024-08-20T22:23:52.8770056Z remote: Counting objects: 33% (575/1741) 2024-08-20T22:23:52.8771826Z remote: Counting objects: 34% (592/1741) 2024-08-20T22:23:52.8772381Z remote: Counting objects: 35% (610/1741) 2024-08-20T22:23:52.8773073Z remote: Counting objects: 36% (627/1741) 2024-08-20T22:23:52.8773594Z remote: Counting objects: 37% (645/1741) 2024-08-20T22:23:52.8774130Z remote: Counting objects: 38% (662/1741) 2024-08-20T22:23:52.8774661Z remote: Counting objects: 39% (679/1741) 2024-08-20T22:23:52.8775187Z remote: Counting objects: 40% (697/1741) 2024-08-20T22:23:52.8775712Z remote: Counting objects: 41% (714/1741) 2024-08-20T22:23:52.8776238Z remote: Counting objects: 42% (732/1741) 2024-08-20T22:23:52.8776773Z remote: Counting objects: 43% (749/1741) 2024-08-20T22:23:52.8777289Z remote: Counting objects: 44% (767/1741) 2024-08-20T22:23:52.8777811Z remote: Counting objects: 45% (784/1741) 2024-08-20T22:23:52.8778335Z remote: Counting objects: 46% (801/1741) 2024-08-20T22:23:52.8778861Z remote: Counting objects: 47% (819/1741) 2024-08-20T22:23:52.8779391Z remote: Counting objects: 48% (836/1741) 2024-08-20T22:23:52.8779917Z remote: Counting objects: 49% (854/1741) 2024-08-20T22:23:52.8780445Z remote: Counting objects: 50% (871/1741) 2024-08-20T22:23:52.8780969Z remote: Counting objects: 51% (888/1741) 2024-08-20T22:23:52.8781489Z remote: Counting objects: 52% (906/1741) 2024-08-20T22:23:52.8782011Z remote: Counting objects: 53% (923/1741) 2024-08-20T22:23:52.8782677Z remote: Counting objects: 54% (941/1741) 2024-08-20T22:23:52.8783211Z remote: Counting objects: 55% (958/1741) 2024-08-20T22:23:52.8783728Z remote: Counting objects: 56% (975/1741) 2024-08-20T22:23:52.8784253Z remote: Counting objects: 57% (993/1741) 2024-08-20T22:23:52.8784787Z remote: Counting objects: 58% (1010/1741) 2024-08-20T22:23:52.8785322Z remote: Counting objects: 59% (1028/1741) 2024-08-20T22:23:52.8785862Z remote: Counting objects: 60% (1045/1741) 2024-08-20T22:23:52.8786407Z remote: Counting objects: 61% (1063/1741) 2024-08-20T22:23:52.8786961Z remote: Counting objects: 62% (1080/1741) 2024-08-20T22:23:52.8787512Z remote: Counting objects: 63% (1097/1741) 2024-08-20T22:23:52.8788052Z remote: Counting objects: 64% (1115/1741) 2024-08-20T22:23:52.8788580Z remote: Counting objects: 65% (1132/1741) 2024-08-20T22:23:52.8789115Z remote: Counting objects: 66% (1150/1741) 2024-08-20T22:23:52.8789643Z remote: Counting objects: 67% (1167/1741) 2024-08-20T22:23:52.8790166Z remote: Counting objects: 68% (1184/1741) 2024-08-20T22:23:52.8790696Z remote: Counting objects: 69% (1202/1741) 2024-08-20T22:23:52.8791228Z remote: Counting objects: 70% (1219/1741) 2024-08-20T22:23:52.8791750Z remote: Counting objects: 71% (1237/1741) 2024-08-20T22:23:52.8792278Z remote: Counting objects: 72% (1254/1741) 2024-08-20T22:23:52.8792806Z remote: Counting objects: 73% (1271/1741) 2024-08-20T22:23:52.8793333Z remote: Counting objects: 74% (1289/1741) 2024-08-20T22:23:52.8793860Z remote: Counting objects: 75% (1306/1741) 2024-08-20T22:23:52.8794389Z remote: Counting objects: 76% (1324/1741) 2024-08-20T22:23:52.8794914Z remote: Counting objects: 77% (1341/1741) 2024-08-20T22:23:52.8795445Z remote: Counting objects: 78% (1358/1741) 2024-08-20T22:23:52.8795968Z remote: Counting objects: 79% (1376/1741) 2024-08-20T22:23:52.8796493Z remote: Counting objects: 80% (1393/1741) 2024-08-20T22:23:52.8797016Z remote: Counting objects: 81% (1411/1741) 2024-08-20T22:23:52.8797589Z remote: Counting objects: 82% (1428/1741) 2024-08-20T22:23:52.8798119Z remote: Counting objects: 83% (1446/1741) 2024-08-20T22:23:52.8798645Z remote: Counting objects: 84% (1463/1741) 2024-08-20T22:23:52.8799175Z remote: Counting objects: 85% (1480/1741) 2024-08-20T22:23:52.8799695Z remote: Counting objects: 86% (1498/1741) 2024-08-20T22:23:52.8800227Z remote: Counting objects: 87% (1515/1741) 2024-08-20T22:23:52.8800758Z remote: Counting objects: 88% (1533/1741) 2024-08-20T22:23:52.8801452Z remote: Counting objects: 89% (1550/1741) 2024-08-20T22:23:52.8801983Z remote: Counting objects: 90% (1567/1741) 2024-08-20T22:23:52.8802597Z remote: Counting objects: 91% (1585/1741) 2024-08-20T22:23:52.8803133Z remote: Counting objects: 92% (1602/1741) 2024-08-20T22:23:52.8803667Z remote: Counting objects: 93% (1620/1741) 2024-08-20T22:23:52.8804189Z remote: Counting objects: 94% (1637/1741) 2024-08-20T22:23:52.8804721Z remote: Counting objects: 95% (1654/1741) 2024-08-20T22:23:52.8805254Z remote: Counting objects: 96% (1672/1741) 2024-08-20T22:23:52.8805781Z remote: Counting objects: 97% (1689/1741) 2024-08-20T22:23:52.8806319Z remote: Counting objects: 98% (1707/1741) 2024-08-20T22:23:52.8806857Z remote: Counting objects: 99% (1724/1741) 2024-08-20T22:23:52.8807383Z remote: Counting objects: 100% (1741/1741) 2024-08-20T22:23:52.8807959Z remote: Counting objects: 100% (1741/1741), done. 2024-08-20T22:23:52.8808540Z remote: Compressing objects: 0% (1/897) 2024-08-20T22:23:52.8846277Z remote: Compressing objects: 1% (9/897) 2024-08-20T22:23:52.8962766Z remote: Compressing objects: 2% (18/897) 2024-08-20T22:23:52.9049319Z remote: Compressing objects: 3% (27/897) 2024-08-20T22:23:52.9096733Z remote: Compressing objects: 4% (36/897) 2024-08-20T22:23:52.9147632Z remote: Compressing objects: 5% (45/897) 2024-08-20T22:23:52.9180278Z remote: Compressing objects: 6% (54/897) 2024-08-20T22:23:52.9211957Z remote: Compressing objects: 7% (63/897) 2024-08-20T22:23:52.9241571Z remote: Compressing objects: 8% (72/897) 2024-08-20T22:23:52.9278640Z remote: Compressing objects: 9% (81/897) 2024-08-20T22:23:52.9304619Z remote: Compressing objects: 10% (90/897) 2024-08-20T22:23:52.9320803Z remote: Compressing objects: 11% (99/897) 2024-08-20T22:23:52.9332490Z remote: Compressing objects: 12% (108/897) 2024-08-20T22:23:52.9348011Z remote: Compressing objects: 13% (117/897) 2024-08-20T22:23:52.9363457Z remote: Compressing objects: 14% (126/897) 2024-08-20T22:23:52.9373049Z remote: Compressing objects: 15% (135/897) 2024-08-20T22:23:52.9381321Z remote: Compressing objects: 16% (144/897) 2024-08-20T22:23:52.9388569Z remote: Compressing objects: 17% (153/897) 2024-08-20T22:23:52.9392547Z remote: Compressing objects: 18% (162/897) 2024-08-20T22:23:52.9395508Z remote: Compressing objects: 19% (171/897) 2024-08-20T22:23:52.9396844Z remote: Compressing objects: 20% (180/897) 2024-08-20T22:23:52.9397820Z remote: Compressing objects: 21% (189/897) 2024-08-20T22:23:52.9398979Z remote: Compressing objects: 22% (198/897) 2024-08-20T22:23:52.9401610Z remote: Compressing objects: 23% (207/897) 2024-08-20T22:23:52.9402458Z remote: Compressing objects: 24% (216/897) 2024-08-20T22:23:52.9404319Z remote: Compressing objects: 25% (225/897) 2024-08-20T22:23:52.9407351Z remote: Compressing objects: 26% (234/897) 2024-08-20T22:23:52.9413824Z remote: Compressing objects: 27% (243/897) 2024-08-20T22:23:52.9416095Z remote: Compressing objects: 28% (252/897) 2024-08-20T22:23:52.9419159Z remote: Compressing objects: 29% (261/897) 2024-08-20T22:23:52.9422348Z remote: Compressing objects: 30% (270/897) 2024-08-20T22:23:52.9424651Z remote: Compressing objects: 31% (279/897) 2024-08-20T22:23:52.9427863Z remote: Compressing objects: 32% (288/897) 2024-08-20T22:23:52.9430817Z remote: Compressing objects: 33% (297/897) 2024-08-20T22:23:52.9433864Z remote: Compressing objects: 34% (305/897) 2024-08-20T22:23:52.9436314Z remote: Compressing objects: 35% (314/897) 2024-08-20T22:23:52.9440429Z remote: Compressing objects: 36% (323/897) 2024-08-20T22:23:52.9440992Z remote: Compressing objects: 37% (332/897) 2024-08-20T22:23:52.9443553Z remote: Compressing objects: 38% (341/897) 2024-08-20T22:23:52.9444517Z remote: Compressing objects: 39% (350/897) 2024-08-20T22:23:52.9446747Z remote: Compressing objects: 40% (359/897) 2024-08-20T22:23:52.9448102Z remote: Compressing objects: 41% (368/897) 2024-08-20T22:23:52.9448703Z remote: Compressing objects: 42% (377/897) 2024-08-20T22:23:52.9449903Z remote: Compressing objects: 43% (386/897) 2024-08-20T22:23:52.9451092Z remote: Compressing objects: 44% (395/897) 2024-08-20T22:23:52.9457094Z remote: Compressing objects: 45% (404/897) 2024-08-20T22:23:52.9457911Z remote: Compressing objects: 46% (413/897) 2024-08-20T22:23:52.9458628Z remote: Compressing objects: 47% (422/897) 2024-08-20T22:23:52.9459190Z remote: Compressing objects: 48% (431/897) 2024-08-20T22:23:52.9459931Z remote: Compressing objects: 49% (440/897) 2024-08-20T22:23:52.9460526Z remote: Compressing objects: 50% (449/897) 2024-08-20T22:23:52.9461257Z remote: Compressing objects: 51% (458/897) 2024-08-20T22:23:52.9461975Z remote: Compressing objects: 52% (467/897) 2024-08-20T22:23:52.9462703Z remote: Compressing objects: 53% (476/897) 2024-08-20T22:23:52.9463270Z remote: Compressing objects: 54% (485/897) 2024-08-20T22:23:52.9463825Z remote: Compressing objects: 55% (494/897) 2024-08-20T22:23:52.9464447Z remote: Compressing objects: 56% (503/897) 2024-08-20T22:23:52.9465200Z remote: Compressing objects: 57% (512/897) 2024-08-20T22:23:52.9465889Z remote: Compressing objects: 58% (521/897) 2024-08-20T22:23:52.9466477Z remote: Compressing objects: 59% (530/897) 2024-08-20T22:23:52.9467039Z remote: Compressing objects: 60% (539/897) 2024-08-20T22:23:52.9467579Z remote: Compressing objects: 61% (548/897) 2024-08-20T22:23:52.9468112Z remote: Compressing objects: 62% (557/897) 2024-08-20T22:23:52.9468650Z remote: Compressing objects: 63% (566/897) 2024-08-20T22:23:52.9469191Z remote: Compressing objects: 64% (575/897) 2024-08-20T22:23:52.9469728Z remote: Compressing objects: 65% (584/897) 2024-08-20T22:23:52.9470257Z remote: Compressing objects: 66% (593/897) 2024-08-20T22:23:52.9470791Z remote: Compressing objects: 67% (601/897) 2024-08-20T22:23:52.9471332Z remote: Compressing objects: 68% (610/897) 2024-08-20T22:23:52.9471868Z remote: Compressing objects: 69% (619/897) 2024-08-20T22:23:52.9472471Z remote: Compressing objects: 70% (628/897) 2024-08-20T22:23:52.9473020Z remote: Compressing objects: 71% (637/897) 2024-08-20T22:23:52.9473562Z remote: Compressing objects: 72% (646/897) 2024-08-20T22:23:52.9474114Z remote: Compressing objects: 73% (655/897) 2024-08-20T22:23:52.9474648Z remote: Compressing objects: 74% (664/897) 2024-08-20T22:23:52.9475190Z remote: Compressing objects: 75% (673/897) 2024-08-20T22:23:52.9475746Z remote: Compressing objects: 76% (682/897) 2024-08-20T22:23:52.9481702Z remote: Compressing objects: 77% (691/897) 2024-08-20T22:23:52.9486345Z remote: Compressing objects: 78% (700/897) 2024-08-20T22:23:52.9491952Z remote: Compressing objects: 79% (709/897) 2024-08-20T22:23:52.9500982Z remote: Compressing objects: 80% (718/897) 2024-08-20T22:23:52.9509230Z remote: Compressing objects: 81% (727/897) 2024-08-20T22:23:52.9515114Z remote: Compressing objects: 82% (736/897) 2024-08-20T22:23:52.9518435Z remote: Compressing objects: 83% (745/897) 2024-08-20T22:23:52.9521950Z remote: Compressing objects: 84% (754/897) 2024-08-20T22:23:52.9525583Z remote: Compressing objects: 85% (763/897) 2024-08-20T22:23:52.9530761Z remote: Compressing objects: 86% (772/897) 2024-08-20T22:23:52.9534637Z remote: Compressing objects: 87% (781/897) 2024-08-20T22:23:52.9539051Z remote: Compressing objects: 88% (790/897) 2024-08-20T22:23:52.9542059Z remote: Compressing objects: 89% (799/897) 2024-08-20T22:23:52.9545655Z remote: Compressing objects: 90% (808/897) 2024-08-20T22:23:52.9549115Z remote: Compressing objects: 91% (817/897) 2024-08-20T22:23:52.9552190Z remote: Compressing objects: 92% (826/897) 2024-08-20T22:23:52.9554986Z remote: Compressing objects: 93% (835/897) 2024-08-20T22:23:52.9557485Z remote: Compressing objects: 94% (844/897) 2024-08-20T22:23:52.9560211Z remote: Compressing objects: 95% (853/897) 2024-08-20T22:23:52.9562818Z remote: Compressing objects: 96% (862/897) 2024-08-20T22:23:52.9565436Z remote: Compressing objects: 97% (871/897) 2024-08-20T22:23:52.9566865Z remote: Compressing objects: 98% (880/897) 2024-08-20T22:23:52.9567667Z remote: Compressing objects: 99% (889/897) 2024-08-20T22:23:52.9569189Z remote: Compressing objects: 100% (897/897) 2024-08-20T22:23:52.9569952Z remote: Compressing objects: 100% (897/897), done. 2024-08-20T22:23:52.9652914Z Receiving objects: 0% (1/31001) 2024-08-20T22:23:52.9696612Z Receiving objects: 1% (311/31001) 2024-08-20T22:23:52.9737730Z Receiving objects: 2% (621/31001) 2024-08-20T22:23:52.9774944Z Receiving objects: 3% (931/31001) 2024-08-20T22:23:52.9808373Z Receiving objects: 4% (1241/31001) 2024-08-20T22:23:52.9841947Z Receiving objects: 5% (1551/31001) 2024-08-20T22:23:52.9879515Z Receiving objects: 6% (1861/31001) 2024-08-20T22:23:52.9918291Z Receiving objects: 7% (2171/31001) 2024-08-20T22:23:52.9949609Z Receiving objects: 8% (2481/31001) 2024-08-20T22:23:52.9982397Z Receiving objects: 9% (2791/31001) 2024-08-20T22:23:53.0053213Z Receiving objects: 10% (3101/31001) 2024-08-20T22:23:53.0102204Z Receiving objects: 11% (3411/31001) 2024-08-20T22:23:53.0168787Z Receiving objects: 12% (3721/31001) 2024-08-20T22:23:53.0300633Z Receiving objects: 13% (4031/31001) 2024-08-20T22:23:53.0446954Z Receiving objects: 14% (4341/31001) 2024-08-20T22:23:53.0515595Z Receiving objects: 15% (4651/31001) 2024-08-20T22:23:53.0591601Z Receiving objects: 16% (4961/31001) 2024-08-20T22:23:53.0648520Z Receiving objects: 17% (5271/31001) 2024-08-20T22:23:53.0680051Z Receiving objects: 18% (5581/31001) 2024-08-20T22:23:53.0739262Z Receiving objects: 19% (5891/31001) 2024-08-20T22:23:53.0773622Z Receiving objects: 20% (6201/31001) 2024-08-20T22:23:53.0790281Z Receiving objects: 21% (6511/31001) 2024-08-20T22:23:53.0838426Z Receiving objects: 22% (6821/31001) 2024-08-20T22:23:53.9595746Z Receiving objects: 23% (7131/31001) 2024-08-20T22:23:54.0694654Z Receiving objects: 23% (7392/31001), 75.07 MiB | 75.14 MiB/s 2024-08-20T22:23:54.6319342Z Receiving objects: 24% (7441/31001), 75.07 MiB | 75.14 MiB/s 2024-08-20T22:23:54.7049406Z Receiving objects: 25% (7751/31001), 112.81 MiB | 75.25 MiB/s 2024-08-20T22:23:54.7769748Z Receiving objects: 26% (8061/31001), 112.81 MiB | 75.25 MiB/s 2024-08-20T22:23:54.8487960Z Receiving objects: 27% (8371/31001), 112.81 MiB | 75.25 MiB/s 2024-08-20T22:23:54.9200028Z Receiving objects: 28% (8681/31001), 112.81 MiB | 75.25 MiB/s 2024-08-20T22:23:54.9595563Z Receiving objects: 29% (8991/31001), 112.81 MiB | 75.25 MiB/s 2024-08-20T22:23:54.9923131Z Receiving objects: 29% (9159/31001), 150.84 MiB | 75.45 MiB/s 2024-08-20T22:23:55.0650241Z Receiving objects: 30% (9301/31001), 150.84 MiB | 75.45 MiB/s 2024-08-20T22:23:55.1365276Z Receiving objects: 31% (9611/31001), 150.84 MiB | 75.45 MiB/s 2024-08-20T22:23:55.3729475Z Receiving objects: 32% (9921/31001), 150.84 MiB | 75.45 MiB/s 2024-08-20T22:23:55.4482726Z Receiving objects: 33% (10231/31001), 150.84 MiB | 75.45 MiB/s 2024-08-20T22:23:55.4605663Z Receiving objects: 34% (10541/31001), 150.84 MiB | 75.45 MiB/s 2024-08-20T22:23:55.7210945Z Receiving objects: 35% (10851/31001), 184.71 MiB | 73.91 MiB/s 2024-08-20T22:23:55.9597341Z Receiving objects: 36% (11161/31001), 184.71 MiB | 73.91 MiB/s 2024-08-20T22:23:55.9705439Z Receiving objects: 36% (11466/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9723867Z Receiving objects: 37% (11471/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9747628Z Receiving objects: 38% (11781/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9765461Z Receiving objects: 39% (12091/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9780270Z Receiving objects: 40% (12401/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9791186Z Receiving objects: 41% (12711/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9803859Z Receiving objects: 42% (13021/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9819620Z Receiving objects: 43% (13331/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9835672Z Receiving objects: 44% (13641/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9844091Z Receiving objects: 45% (13951/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9848925Z Receiving objects: 46% (14261/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9863492Z Receiving objects: 47% (14571/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9884419Z Receiving objects: 48% (14881/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:55.9897117Z Receiving objects: 49% (15191/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0097678Z Receiving objects: 50% (15501/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0102874Z Receiving objects: 51% (15811/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0109813Z Receiving objects: 52% (16121/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0116011Z Receiving objects: 53% (16431/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0121403Z Receiving objects: 54% (16741/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0138460Z Receiving objects: 55% (17051/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0149622Z Receiving objects: 56% (17361/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0155832Z Receiving objects: 57% (17671/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0169430Z Receiving objects: 58% (17981/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0181436Z Receiving objects: 59% (18291/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0191952Z Receiving objects: 60% (18601/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0329414Z Receiving objects: 61% (18911/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0335059Z Receiving objects: 62% (19221/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0341608Z Receiving objects: 63% (19531/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0349891Z Receiving objects: 64% (19841/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0357468Z Receiving objects: 65% (20151/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0365946Z Receiving objects: 66% (20461/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.0621781Z Receiving objects: 67% (20771/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1020267Z Receiving objects: 68% (21081/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1027625Z Receiving objects: 69% (21391/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1042194Z Receiving objects: 70% (21701/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1053132Z Receiving objects: 71% (22011/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1067985Z Receiving objects: 72% (22321/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1077212Z Receiving objects: 73% (22631/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1092352Z Receiving objects: 74% (22941/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1103344Z Receiving objects: 75% (23251/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1110585Z Receiving objects: 76% (23561/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1118620Z Receiving objects: 77% (23871/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1332714Z Receiving objects: 78% (24181/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1366010Z Receiving objects: 79% (24491/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1375267Z Receiving objects: 80% (24801/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1387554Z Receiving objects: 81% (25111/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1399817Z Receiving objects: 82% (25421/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1448298Z Receiving objects: 83% (25731/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1586653Z Receiving objects: 84% (26041/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1591082Z Receiving objects: 85% (26351/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1599162Z Receiving objects: 86% (26661/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1604333Z Receiving objects: 87% (26971/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1611587Z Receiving objects: 88% (27281/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1617494Z Receiving objects: 89% (27591/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1729354Z Receiving objects: 90% (27901/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1744352Z Receiving objects: 91% (28211/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1755931Z Receiving objects: 92% (28521/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1773092Z Receiving objects: 93% (28831/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1806159Z Receiving objects: 94% (29141/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1818337Z Receiving objects: 95% (29451/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1823976Z Receiving objects: 96% (29761/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1831836Z Receiving objects: 97% (30071/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1906171Z Receiving objects: 98% (30381/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1926200Z Receiving objects: 99% (30691/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1927310Z remote: Total 31001 (delta 1016), reused 1474 (delta 825), pack-reused 29260 (from 1) 2024-08-20T22:23:56.1944085Z Receiving objects: 100% (31001/31001), 221.22 MiB | 73.76 MiB/s 2024-08-20T22:23:56.1944785Z Receiving objects: 100% (31001/31001), 232.50 MiB | 71.88 MiB/s, done. 2024-08-20T22:23:56.1975759Z Resolving deltas: 0% (0/16729) 2024-08-20T22:23:56.1990361Z Resolving deltas: 1% (168/16729) 2024-08-20T22:23:56.2000337Z Resolving deltas: 2% (403/16729) 2024-08-20T22:23:56.2023870Z Resolving deltas: 3% (516/16729) 2024-08-20T22:23:56.2039723Z Resolving deltas: 4% (671/16729) 2024-08-20T22:23:56.2049429Z Resolving deltas: 5% (899/16729) 2024-08-20T22:23:56.2058954Z Resolving deltas: 6% (1024/16729) 2024-08-20T22:23:56.2071798Z Resolving deltas: 7% (1172/16729) 2024-08-20T22:23:56.2081311Z Resolving deltas: 8% (1349/16729) 2024-08-20T22:23:56.2093403Z Resolving deltas: 9% (1515/16729) 2024-08-20T22:23:56.2103839Z Resolving deltas: 10% (1677/16729) 2024-08-20T22:23:56.2117545Z Resolving deltas: 11% (1842/16729) 2024-08-20T22:23:56.2129464Z Resolving deltas: 12% (2014/16729) 2024-08-20T22:23:56.2145420Z Resolving deltas: 13% (2177/16729) 2024-08-20T22:23:56.2160081Z Resolving deltas: 14% (2416/16729) 2024-08-20T22:23:56.2175537Z Resolving deltas: 15% (2513/16729) 2024-08-20T22:23:56.2199201Z Resolving deltas: 16% (2683/16729) 2024-08-20T22:23:56.2202192Z Resolving deltas: 17% (2967/16729) 2024-08-20T22:23:56.2211345Z Resolving deltas: 18% (3017/16729) 2024-08-20T22:23:56.2219850Z Resolving deltas: 19% (3181/16729) 2024-08-20T22:23:56.2256225Z Resolving deltas: 20% (3355/16729) 2024-08-20T22:23:56.2283806Z Resolving deltas: 21% (3525/16729) 2024-08-20T22:23:56.2292583Z Resolving deltas: 22% (3780/16729) 2024-08-20T22:23:56.2303220Z Resolving deltas: 23% (3908/16729) 2024-08-20T22:23:56.2321661Z Resolving deltas: 24% (4020/16729) 2024-08-20T22:23:56.2330302Z Resolving deltas: 25% (4250/16729) 2024-08-20T22:23:56.2342255Z Resolving deltas: 26% (4387/16729) 2024-08-20T22:23:56.2380160Z Resolving deltas: 27% (4541/16729) 2024-08-20T22:23:56.2399613Z Resolving deltas: 28% (4685/16729) 2024-08-20T22:23:56.2415298Z Resolving deltas: 29% (4853/16729) 2024-08-20T22:23:56.2434510Z Resolving deltas: 30% (5019/16729) 2024-08-20T22:23:56.2450234Z Resolving deltas: 31% (5199/16729) 2024-08-20T22:23:56.2466263Z Resolving deltas: 32% (5361/16729) 2024-08-20T22:23:56.2477171Z Resolving deltas: 33% (5549/16729) 2024-08-20T22:23:56.2485142Z Resolving deltas: 34% (5730/16729) 2024-08-20T22:23:56.2500187Z Resolving deltas: 35% (5859/16729) 2024-08-20T22:23:56.2521174Z Resolving deltas: 36% (6023/16729) 2024-08-20T22:23:56.2538500Z Resolving deltas: 37% (6191/16729) 2024-08-20T22:23:56.2552900Z Resolving deltas: 38% (6360/16729) 2024-08-20T22:23:56.2565300Z Resolving deltas: 39% (6554/16729) 2024-08-20T22:23:56.2585657Z Resolving deltas: 40% (6692/16729) 2024-08-20T22:23:56.2598490Z Resolving deltas: 41% (6879/16729) 2024-08-20T22:23:56.2613509Z Resolving deltas: 42% (7056/16729) 2024-08-20T22:23:56.2627655Z Resolving deltas: 43% (7211/16729) 2024-08-20T22:23:56.2637447Z Resolving deltas: 44% (7361/16729) 2024-08-20T22:23:56.2648318Z Resolving deltas: 45% (7546/16729) 2024-08-20T22:23:56.2665025Z Resolving deltas: 46% (7696/16729) 2024-08-20T22:23:56.2688091Z Resolving deltas: 47% (7863/16729) 2024-08-20T22:23:56.2704108Z Resolving deltas: 48% (8038/16729) 2024-08-20T22:23:56.2722309Z Resolving deltas: 49% (8228/16729) 2024-08-20T22:23:56.2731755Z Resolving deltas: 50% (8365/16729) 2024-08-20T22:23:56.2740755Z Resolving deltas: 51% (8534/16729) 2024-08-20T22:23:56.2752483Z Resolving deltas: 52% (8700/16729) 2024-08-20T22:23:56.2768604Z Resolving deltas: 53% (8882/16729) 2024-08-20T22:23:56.2779427Z Resolving deltas: 54% (9047/16729) 2024-08-20T22:23:56.2795347Z Resolving deltas: 55% (9202/16729) 2024-08-20T22:23:56.2803948Z Resolving deltas: 56% (9371/16729) 2024-08-20T22:23:56.2813799Z Resolving deltas: 57% (9568/16729) 2024-08-20T22:23:56.2820623Z Resolving deltas: 58% (9796/16729) 2024-08-20T22:23:56.2835752Z Resolving deltas: 59% (9902/16729) 2024-08-20T22:23:56.2842685Z Resolving deltas: 60% (10105/16729) 2024-08-20T22:23:56.2856112Z Resolving deltas: 61% (10216/16729) 2024-08-20T22:23:56.2862329Z Resolving deltas: 62% (10430/16729) 2024-08-20T22:23:56.2892069Z Resolving deltas: 63% (10540/16729) 2024-08-20T22:23:56.2912598Z Resolving deltas: 65% (10919/16729) 2024-08-20T22:23:56.2914197Z Resolving deltas: 66% (11191/16729) 2024-08-20T22:23:56.2934824Z Resolving deltas: 67% (11210/16729) 2024-08-20T22:23:56.2945094Z Resolving deltas: 68% (11377/16729) 2024-08-20T22:23:56.2957504Z Resolving deltas: 69% (11558/16729) 2024-08-20T22:23:56.2974739Z Resolving deltas: 70% (11719/16729) 2024-08-20T22:23:56.2989529Z Resolving deltas: 71% (11879/16729) 2024-08-20T22:23:56.2998502Z Resolving deltas: 72% (12078/16729) 2024-08-20T22:23:56.3018472Z Resolving deltas: 73% (12214/16729) 2024-08-20T22:23:56.3022935Z Resolving deltas: 74% (12496/16729) 2024-08-20T22:23:56.3031568Z Resolving deltas: 75% (12576/16729) 2024-08-20T22:23:56.3040242Z Resolving deltas: 76% (12749/16729) 2024-08-20T22:23:56.3048684Z Resolving deltas: 77% (12893/16729) 2024-08-20T22:23:56.3061064Z Resolving deltas: 78% (13050/16729) 2024-08-20T22:23:56.3071103Z Resolving deltas: 79% (13221/16729) 2024-08-20T22:23:56.3080218Z Resolving deltas: 80% (13384/16729) 2024-08-20T22:23:56.3091002Z Resolving deltas: 81% (13563/16729) 2024-08-20T22:23:56.3100723Z Resolving deltas: 82% (13739/16729) 2024-08-20T22:23:56.3110359Z Resolving deltas: 83% (13887/16729) 2024-08-20T22:23:56.3116792Z Resolving deltas: 84% (14087/16729) 2024-08-20T22:23:56.3125327Z Resolving deltas: 85% (14221/16729) 2024-08-20T22:23:56.3135390Z Resolving deltas: 86% (14392/16729) 2024-08-20T22:23:56.3144362Z Resolving deltas: 87% (14570/16729) 2024-08-20T22:23:56.3154443Z Resolving deltas: 88% (14722/16729) 2024-08-20T22:23:56.3161371Z Resolving deltas: 89% (14900/16729) 2024-08-20T22:23:56.3169629Z Resolving deltas: 90% (15072/16729) 2024-08-20T22:23:56.3178165Z Resolving deltas: 91% (15224/16729) 2024-08-20T22:23:56.3185913Z Resolving deltas: 92% (15392/16729) 2024-08-20T22:23:56.3218391Z Resolving deltas: 93% (15563/16729) 2024-08-20T22:23:56.3219004Z Resolving deltas: 94% (15731/16729) 2024-08-20T22:23:56.3224655Z Resolving deltas: 95% (15946/16729) 2024-08-20T22:23:56.3231476Z Resolving deltas: 96% (16193/16729) 2024-08-20T22:23:56.3240181Z Resolving deltas: 97% (16256/16729) 2024-08-20T22:23:56.3290492Z Resolving deltas: 98% (16397/16729) 2024-08-20T22:23:56.3366508Z Resolving deltas: 99% (16567/16729) 2024-08-20T22:23:56.3367025Z Resolving deltas: 100% (16729/16729) 2024-08-20T22:23:56.3367468Z Resolving deltas: 100% (16729/16729), done. 2024-08-20T22:23:57.3356638Z + pushd torchbench 2024-08-20T22:23:57.3357099Z ~/workspace/torchbench ~/workspace 2024-08-20T22:23:57.3357650Z + git checkout 23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T22:23:57.4140604Z Note: switching to '23512dbebd44a11eb84afbf53c3c071dd105297e'. 2024-08-20T22:23:57.4141098Z 2024-08-20T22:23:57.4141642Z You are in 'detached HEAD' state. You can look around, make experimental 2024-08-20T22:23:57.4142768Z changes and commit them, and you can discard any commits you make in this 2024-08-20T22:23:57.4143518Z state without impacting any branches by switching back to a branch. 2024-08-20T22:23:57.4143954Z 2024-08-20T22:23:57.4144270Z If you want to create a new branch to retain commits you create, you may 2024-08-20T22:23:57.4145074Z do so (now or later) by using -c with the switch command. Example: 2024-08-20T22:23:57.4145488Z 2024-08-20T22:23:57.4145688Z git switch -c 2024-08-20T22:23:57.4145953Z 2024-08-20T22:23:57.4146089Z Or undo this operation with: 2024-08-20T22:23:57.4146353Z 2024-08-20T22:23:57.4146479Z git switch - 2024-08-20T22:23:57.4146666Z 2024-08-20T22:23:57.4146996Z Turn off this advice by setting config variable advice.detachedHead to false 2024-08-20T22:23:57.4147491Z 2024-08-20T22:23:57.4147666Z HEAD is now at 23512dbe Pin numpy version 2024-08-20T22:23:57.4153814Z + '[' '' ']' 2024-08-20T22:23:57.4154200Z + python install.py --continue_on_fail 2024-08-20T22:24:00.0762308Z checking packages torch, torchvision, torchaudio are installed...OK 2024-08-20T22:24:18.1435908Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch...OK 2024-08-20T22:24:21.8076389Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Background_Matting...OK 2024-08-20T22:24:23.6960941Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/LearningToPaint...OK 2024-08-20T22:24:25.1962821Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Super_SloMo...OK 2024-08-20T22:24:25.2201509Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/alexnet...OK 2024-08-20T22:24:32.6640054Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_edgecnn...OK 2024-08-20T22:24:35.3657249Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gcn...OK 2024-08-20T22:24:38.0632037Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gin...OK 2024-08-20T22:24:40.7617575Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_sage...OK 2024-08-20T22:24:40.7621094Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/cm3leon_generate...SKIP - No install.py is found 2024-08-20T22:24:41.8054366Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dcgan...OK 2024-08-20T22:24:44.5906089Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/demucs...OK 2024-08-20T22:24:44.6141293Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/densenet121...OK 2024-08-20T22:25:40.6962951Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_c4...OK 2024-08-20T22:25:49.2789227Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_dc5...OK 2024-08-20T22:25:56.4116879Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_fpn...OK 2024-08-20T22:26:03.1072651Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_c4...OK 2024-08-20T22:26:11.1982226Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_dc5...OK 2024-08-20T22:26:20.9187301Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_fpn...OK 2024-08-20T22:26:27.2490831Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fcos_r_50_fpn...OK 2024-08-20T22:26:34.1677002Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn...OK 2024-08-20T22:26:41.4444909Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_c4...OK 2024-08-20T22:26:48.5906313Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_fpn...OK 2024-08-20T22:26:55.3547688Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_c4...OK 2024-08-20T22:27:02.2209636Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_fpn...OK 2024-08-20T22:27:03.4028103Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dlrm...OK 2024-08-20T22:27:38.0741641Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_det_predictor...OK 2024-08-20T22:27:42.7979822Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_reco_predictor...OK 2024-08-20T22:27:47.9551807Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/drq...OK 2024-08-20T22:27:58.8954173Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/fastNLP_Bert...OK 2024-08-20T22:28:00.0745474Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_dp_cifar10...OK 2024-08-20T22:28:01.5309032Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_maml_omniglot...OK 2024-08-20T22:28:08.0988433Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Albert...OK 2024-08-20T22:28:15.1407530Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bart...OK 2024-08-20T22:28:20.5903511Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert...OK 2024-08-20T22:28:28.2661585Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert_large...OK 2024-08-20T22:28:34.1596890Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_BigBird...OK 2024-08-20T22:28:39.1401189Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_DistilBert...OK 2024-08-20T22:28:45.3222371Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2...OK 2024-08-20T22:29:01.7258506Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2_large...OK 2024-08-20T22:29:07.9256100Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Longformer...OK 2024-08-20T22:29:11.8257396Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Reformer...OK 2024-08-20T22:29:16.5548885Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5...OK 2024-08-20T22:29:23.2953422Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_base...OK 2024-08-20T22:29:23.2957186Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_generate...SKIP - No install.py is found 2024-08-20T22:29:36.3803960Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_large...OK 2024-08-20T22:29:40.3029495Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Whisper...OK 2024-08-20T22:29:40.3032672Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_clip...SKIP - No install.py is found 2024-08-20T22:29:47.8721831Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_distil_whisper...OK 2024-08-20T22:29:49.0934020Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/lennard_jones...OK 2024-08-20T22:29:50.3475895Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama...OK 2024-08-20T22:30:35.1947168Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama_v2_7b_16h...OK 2024-08-20T22:32:04.8769973Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llava...OK 2024-08-20T22:32:04.9014766Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml...OK 2024-08-20T22:32:06.4031701Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml_omniglot...OK 2024-08-20T22:32:06.4270694Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mnasnet1_0...OK 2024-08-20T22:32:06.4514368Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2...OK 2024-08-20T22:32:06.4755692Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2_quantized_qat...OK 2024-08-20T22:32:06.4992700Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v3_large...OK 2024-08-20T22:32:06.5232730Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco...OK 2024-08-20T22:32:27.6740044Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moondream...OK 2024-08-20T22:32:27.6743110Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nanogpt...SKIP - No install.py is found 2024-08-20T22:32:28.9371016Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nvidia_deeprecommender...OK 2024-08-20T22:32:31.3236204Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/opacus_cifar10...OK 2024-08-20T22:32:31.3483963Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_densenet...OK 2024-08-20T22:32:31.3725131Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_resnet...OK 2024-08-20T22:32:31.3981344Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_equation_of_state...OK 2024-08-20T22:32:31.4228868Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_isoneutral_mixing...OK 2024-08-20T22:32:31.4469520Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_turbulent_kinetic_energy...OK 2024-08-20T22:32:38.0191848Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_CycleGAN_and_pix2pix...OK 2024-08-20T22:32:39.4660685Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_stargan...OK 2024-08-20T22:32:42.9297136Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_unet...OK 2024-08-20T22:32:42.9541001Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet152...OK 2024-08-20T22:32:42.9789414Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet18...OK 2024-08-20T22:32:43.0035757Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50...OK 2024-08-20T22:32:43.0278991Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50_quantized_qat...OK 2024-08-20T22:32:43.0517535Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnext50_32x4d...OK 2024-08-20T22:32:57.7466829Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam...OK 2024-08-20T22:33:14.6983383Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam_fast...OK 2024-08-20T22:33:14.7231097Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/shufflenet_v2_x1_0...OK 2024-08-20T22:33:14.7235919Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt...SKIP - No install.py is found 2024-08-20T22:33:14.7237782Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt_tp_manual...SKIP - No install.py is found 2024-08-20T22:33:17.1606665Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/soft_actor_critic...OK 2024-08-20T22:33:18.9443316Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/speech_transformer...OK 2024-08-20T22:33:18.9689754Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/squeezenet1_1...OK 2024-08-20T22:33:43.7670614Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_text_encoder...OK 2024-08-20T22:33:50.3653121Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_unet...OK 2024-08-20T22:33:55.1379010Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tacotron2...OK 2024-08-20T22:34:01.5362239Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientdet...OK 2024-08-20T22:34:01.5606374Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientnet...OK 2024-08-20T22:34:01.5849921Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_nfnet...OK 2024-08-20T22:34:01.6098408Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_regnet...OK 2024-08-20T22:34:01.6341840Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_resnest...OK 2024-08-20T22:34:01.6581891Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer...OK 2024-08-20T22:34:01.6825325Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer_large...OK 2024-08-20T22:34:01.7067025Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vovnet...OK 2024-08-20T22:34:11.2720436Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/torch_multimodal_clip...OK 2024-08-20T22:34:23.5501554Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tts_angular...OK 2024-08-20T22:34:23.5757418Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vgg16...OK 2024-08-20T22:34:25.6598231Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vision_maskrcnn...OK 2024-08-20T22:34:27.6475539Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/yolov3...OK 2024-08-20T22:34:28.1242431Z + echo 'Print all dependencies after TorchBench is installed' 2024-08-20T22:34:28.1243199Z Print all dependencies after TorchBench is installed 2024-08-20T22:34:28.1245700Z + python -mpip freeze 2024-08-20T22:34:28.6301919Z absl-py==2.1.0 2024-08-20T22:34:28.6302368Z accelerate==0.33.0 2024-08-20T22:34:28.6302970Z aiohappyeyeballs==2.4.0 2024-08-20T22:34:28.6303356Z aiohttp==3.10.5 2024-08-20T22:34:28.6306091Z aiosignal==1.3.1 2024-08-20T22:34:28.6306843Z annotated-types==0.7.0 2024-08-20T22:34:28.6307449Z antlr4-python3-runtime==4.9.3 2024-08-20T22:34:28.6307946Z anyascii==0.3.2 2024-08-20T22:34:28.6308351Z astroid==3.2.4 2024-08-20T22:34:28.6308705Z astunparse==1.6.3 2024-08-20T22:34:28.6309146Z async-timeout==4.0.3 2024-08-20T22:34:28.6309557Z attrs==23.1.0 2024-08-20T22:34:28.6309857Z audioread==3.0.1 2024-08-20T22:34:28.6310165Z babel==2.16.0 2024-08-20T22:34:28.6310547Z beautifulsoup4==4.12.3 2024-08-20T22:34:28.6311923Z -e git+https://github.com/pytorch/benchmark@23512dbebd44a11eb84afbf53c3c071dd105297e#egg=bert_pytorch&subdirectory=torchbenchmark/models/BERT_pytorch 2024-08-20T22:34:28.6313003Z bibtexparser==2.0.0b7 2024-08-20T22:34:28.6313365Z black==24.8.0 2024-08-20T22:34:28.6313659Z blinker==1.8.2 2024-08-20T22:34:28.6313958Z blis==0.7.11 2024-08-20T22:34:28.6314253Z blobfile==2.1.1 2024-08-20T22:34:28.6314548Z bokeh==3.5.1 2024-08-20T22:34:28.6314856Z boto3==1.19.12 2024-08-20T22:34:28.6315227Z botocore==1.22.12 2024-08-20T22:34:28.6315529Z bs4==0.0.2 2024-08-20T22:34:28.6315882Z cachetools==5.5.0 2024-08-20T22:34:28.6316249Z cardboardlint==1.3.1 2024-08-20T22:34:28.6316652Z catalogue==2.0.10 2024-08-20T22:34:28.6317081Z certifi==2024.7.4 2024-08-20T22:34:28.6317464Z cffi==1.17.0 2024-08-20T22:34:28.6318351Z charset-normalizer==3.3.2 2024-08-20T22:34:28.6318840Z click==8.1.7 2024-08-20T22:34:28.6319207Z clldutils==3.22.2 2024-08-20T22:34:28.6319617Z cloudpathlib==0.18.1 2024-08-20T22:34:28.6320399Z cloudpickle==3.0.0 2024-08-20T22:34:28.6320810Z colorama==0.4.6 2024-08-20T22:34:28.6321202Z colorlog==6.8.2 2024-08-20T22:34:28.6321512Z confection==0.1.5 2024-08-20T22:34:28.6321817Z contourpy==1.2.1 2024-08-20T22:34:28.6322205Z coremltools==5.0b5 2024-08-20T22:34:28.6322619Z cryptography==43.0.0 2024-08-20T22:34:28.6323034Z csvw==3.3.0 2024-08-20T22:34:28.6323406Z cycler==0.12.1 2024-08-20T22:34:28.6323802Z cymem==2.0.8 2024-08-20T22:34:28.6324255Z DALL-E==0.1 2024-08-20T22:34:28.6324574Z datasets==2.21.0 2024-08-20T22:34:28.6324884Z decorator==5.1.1 2024-08-20T22:34:28.6325196Z defusedxml==0.7.1 2024-08-20T22:34:28.6325500Z Deprecated==1.2.14 2024-08-20T22:34:28.6326216Z detectron2 @ git+https://github.com/facebookresearch/detectron2.git@1a4df4d954089534daebbd7a0a45f08ece87056e 2024-08-20T22:34:28.6326998Z diffusers==0.20.2 2024-08-20T22:34:28.6327307Z dill==0.3.8 2024-08-20T22:34:28.6327584Z diskcache==5.6.3 2024-08-20T22:34:28.6327889Z distro==1.9.0 2024-08-20T22:34:28.6328189Z dlinfo==1.2.1 2024-08-20T22:34:28.6328540Z docker-pycreds==0.4.0 2024-08-20T22:34:28.6328864Z dominate==2.9.1 2024-08-20T22:34:28.6329671Z effdet @ git+https://github.com/rwightman/efficientdet-pytorch.git@d43c9e34cd62d22b4205831bb735f6dd83b8e881 2024-08-20T22:34:28.6330446Z einops==0.8.0 2024-08-20T22:34:28.6330744Z exceptiongroup==1.2.2 2024-08-20T22:34:28.6331077Z execnet==2.1.1 2024-08-20T22:34:28.6331384Z expecttest==0.1.6 2024-08-20T22:34:28.6331736Z FastNLP==0.6.0 2024-08-20T22:34:28.6332078Z ffmpeg-python==0.2.0 2024-08-20T22:34:28.6332413Z filelock==3.13.1 2024-08-20T22:34:28.6332717Z Flask==3.0.3 2024-08-20T22:34:28.6333018Z flatbuffers==2.0 2024-08-20T22:34:28.6333319Z fonttools==4.53.1 2024-08-20T22:34:28.6333636Z frozenlist==1.4.1 2024-08-20T22:34:28.6333943Z fsspec==2024.6.1 2024-08-20T22:34:28.6334235Z ftfy==6.2.3 2024-08-20T22:34:28.6334850Z functorch @ git+https://github.com/pytorch/functorch.git@b71aa0b4387b86c278132209b99538be48ef4c74 2024-08-20T22:34:28.6335572Z future==1.0.0 2024-08-20T22:34:28.6335887Z fvcore==0.1.5.post20221221 2024-08-20T22:34:28.6336228Z gdown==5.2.0 2024-08-20T22:34:28.6336917Z geojson==2.5.0 2024-08-20T22:34:28.6337228Z ghstack==0.8.0 2024-08-20T22:34:28.6337530Z gitdb==4.0.11 2024-08-20T22:34:28.6337838Z GitPython==3.1.43 2024-08-20T22:34:28.6338217Z google-auth==2.34.0 2024-08-20T22:34:28.6338606Z google-auth-oauthlib==1.0.0 2024-08-20T22:34:28.6338974Z grpcio==1.65.5 2024-08-20T22:34:28.6339277Z gym==0.26.2 2024-08-20T22:34:28.6339608Z gym-notices==0.0.8 2024-08-20T22:34:28.6339927Z h5py==3.11.0 2024-08-20T22:34:28.6340227Z higher==0.2.1 2024-08-20T22:34:28.6340580Z huggingface-hub==0.24.6 2024-08-20T22:34:28.6341014Z hydra-core==1.3.2 2024-08-20T22:34:28.6341320Z hypothesis==5.35.1 2024-08-20T22:34:28.6341625Z idna==3.7 2024-08-20T22:34:28.6341900Z imageio==2.35.1 2024-08-20T22:34:28.6342217Z importlib_metadata==8.4.0 2024-08-20T22:34:28.6342657Z inflect==7.3.1 2024-08-20T22:34:28.6342969Z iniconfig==2.0.0 2024-08-20T22:34:28.6343284Z iopath==0.1.9 2024-08-20T22:34:28.6343581Z isodate==0.6.1 2024-08-20T22:34:28.6343880Z isort==5.13.2 2024-08-20T22:34:28.6344182Z itsdangerous==2.2.0 2024-08-20T22:34:28.6344503Z Jinja2==3.1.4 2024-08-20T22:34:28.6344795Z jmespath==0.10.0 2024-08-20T22:34:28.6345103Z joblib==1.4.2 2024-08-20T22:34:28.6345403Z jsonpatch==1.33 2024-08-20T22:34:28.6345711Z jsonpointer==3.0.0 2024-08-20T22:34:28.6346035Z jsonschema==4.23.0 2024-08-20T22:34:28.6346446Z jsonschema-specifications==2023.12.1 2024-08-20T22:34:28.6346867Z junitparser==2.1.1 2024-08-20T22:34:28.6347216Z kaldi-io==0.9.8 2024-08-20T22:34:28.6347521Z kiwisolver==1.4.5 2024-08-20T22:34:28.6347831Z kornia==0.7.3 2024-08-20T22:34:28.6348126Z kornia_rs==0.1.5 2024-08-20T22:34:28.6348434Z lameenc==1.7.0 2024-08-20T22:34:28.6348734Z langcodes==3.4.0 2024-08-20T22:34:28.6349045Z langdetect==1.0.9 2024-08-20T22:34:28.6349555Z language-tags==1.2.0 2024-08-20T22:34:28.6349903Z language_data==1.2.0 2024-08-20T22:34:28.6350237Z lark==0.12.0 2024-08-20T22:34:28.6350531Z lazy_loader==0.4 2024-08-20T22:34:28.6350952Z libcst==1.4.0 2024-08-20T22:34:28.6351254Z librosa==0.9.2 2024-08-20T22:34:28.6351564Z lintrunner==0.12.5 2024-08-20T22:34:28.6351880Z llvmlite==0.38.1 2024-08-20T22:34:28.6352194Z lxml==4.9.4 2024-08-20T22:34:28.6352534Z marisa-trie==1.2.0 2024-08-20T22:34:28.6352849Z Markdown==3.7 2024-08-20T22:34:28.6353198Z markdown-it-py==3.0.0 2024-08-20T22:34:28.6353546Z MarkupSafe==2.1.5 2024-08-20T22:34:28.6353870Z matplotlib==3.8.4 2024-08-20T22:34:28.6354179Z mccabe==0.7.0 2024-08-20T22:34:28.6354480Z mdurl==0.1.2 2024-08-20T22:34:28.6354818Z ml-dtypes==0.4.0 2024-08-20T22:34:28.6355130Z MonkeyType==23.3.0 2024-08-20T22:34:28.6355499Z more-itertools==10.4.0 2024-08-20T22:34:28.6355842Z mpmath==1.3.0 2024-08-20T22:34:28.6356147Z msgpack==1.0.8 2024-08-20T22:34:28.6356450Z multidict==6.0.5 2024-08-20T22:34:28.6356780Z multiprocess==0.70.16 2024-08-20T22:34:28.6357117Z murmurhash==1.0.10 2024-08-20T22:34:28.6357421Z musdb==0.4.2 2024-08-20T22:34:28.6357721Z museval==0.4.1 2024-08-20T22:34:28.6358025Z mypy==1.10.0 2024-08-20T22:34:28.6358370Z mypy-extensions==1.0.0 2024-08-20T22:34:28.6358703Z networkx==2.8.8 2024-08-20T22:34:28.6359005Z ninja==1.10.2 2024-08-20T22:34:28.6359297Z nose==1.3.7 2024-08-20T22:34:28.6359576Z numba==0.55.2 2024-08-20T22:34:28.6359870Z numpy==1.21.2 2024-08-20T22:34:28.6360279Z nvidia-cublas-cu12==12.1.3.1 2024-08-20T22:34:28.6360789Z nvidia-cuda-cupti-cu12==12.1.105 2024-08-20T22:34:28.6361241Z nvidia-cuda-nvrtc-cu12==12.1.105 2024-08-20T22:34:28.6361701Z nvidia-cuda-runtime-cu12==12.1.105 2024-08-20T22:34:28.6362150Z nvidia-cudnn-cu12==9.1.0.70 2024-08-20T22:34:28.6362567Z nvidia-cufft-cu12==11.0.2.54 2024-08-20T22:34:28.6362979Z nvidia-curand-cu12==10.3.2.106 2024-08-20T22:34:28.6363414Z nvidia-cusolver-cu12==11.4.5.107 2024-08-20T22:34:28.6363858Z nvidia-cusparse-cu12==12.1.0.106 2024-08-20T22:34:28.6364289Z nvidia-nccl-cu12==2.20.5 2024-08-20T22:34:28.6364697Z nvidia-nvjitlink-cu12==12.6.20 2024-08-20T22:34:28.6365122Z nvidia-nvtx-cu12==12.1.105 2024-08-20T22:34:28.6365478Z oauthlib==3.2.2 2024-08-20T22:34:28.6365776Z omegaconf==2.3.0 2024-08-20T22:34:28.6366079Z onnx==1.16.1 2024-08-20T22:34:28.6366390Z onnxscript==0.1.0.dev20240817 2024-08-20T22:34:28.6366755Z opacus==1.5.2 2024-08-20T22:34:28.6367098Z opencv-python==4.8.0.74 2024-08-20T22:34:28.6367477Z opt-einsum==3.3.0 2024-08-20T22:34:28.6367787Z optree==0.12.1 2024-08-20T22:34:28.6368086Z packaging==24.1 2024-08-20T22:34:28.6368389Z pandas==2.0.3 2024-08-20T22:34:28.6368677Z patch==1.16 2024-08-20T22:34:28.6368963Z pathspec==0.12.1 2024-08-20T22:34:28.6369266Z phonemizer==3.3.0 2024-08-20T22:34:28.6369574Z pillow==10.3.0 2024-08-20T22:34:28.6369874Z platformdirs==4.2.2 2024-08-20T22:34:28.6370183Z pluggy==1.5.0 2024-08-20T22:34:28.6370474Z pooch==1.8.2 2024-08-20T22:34:28.6370765Z portalocker==2.10.1 2024-08-20T22:34:28.6371087Z preshed==3.0.9 2024-08-20T22:34:28.6371380Z prettytable==3.11.0 2024-08-20T22:34:28.6371695Z protobuf==3.20.2 2024-08-20T22:34:28.6371994Z psutil==6.0.0 2024-08-20T22:34:28.6372327Z py-cpuinfo==9.0.0 2024-08-20T22:34:28.6372637Z pyaml==24.7.0 2024-08-20T22:34:28.6372924Z pyarrow==17.0.0 2024-08-20T22:34:28.6373224Z pyasn1==0.6.0 2024-08-20T22:34:28.6373515Z pyasn1_modules==0.4.0 2024-08-20T22:34:28.6373853Z pyclipper==1.3.0.post5 2024-08-20T22:34:28.6374190Z pycocotools==2.0.8 2024-08-20T22:34:28.6374501Z pycparser==2.22 2024-08-20T22:34:28.6374812Z pycryptodomex==3.20.0 2024-08-20T22:34:28.6375142Z pydantic==2.8.2 2024-08-20T22:34:28.6375452Z pydantic_core==2.20.1 2024-08-20T22:34:28.6375773Z pydot==3.0.1 2024-08-20T22:34:28.6376065Z pygame==2.6.0 2024-08-20T22:34:28.6376361Z PyGithub==2.3.0 2024-08-20T22:34:28.6376661Z Pygments==2.15.0 2024-08-20T22:34:28.6376966Z PyJWT==2.9.0 2024-08-20T22:34:28.6377259Z pylatexenc==2.10 2024-08-20T22:34:28.6377565Z pylint==3.2.6 2024-08-20T22:34:28.6377846Z PyNaCl==1.5.0 2024-08-20T22:34:28.6378238Z pynvml==11.5.3 2024-08-20T22:34:28.6378552Z pyparsing==3.1.2 2024-08-20T22:34:28.6378859Z pypdfium2==4.30.0 2024-08-20T22:34:28.6379171Z pysbd==0.3.4 2024-08-20T22:34:28.6379544Z PySocks==1.7.1 2024-08-20T22:34:28.6379842Z pytest==7.3.2 2024-08-20T22:34:28.6380189Z pytest-benchmark==4.0.0 2024-08-20T22:34:28.6380572Z pytest-cpp==2.3.0 2024-08-20T22:34:28.6380943Z pytest-flakefinder==1.1.0 2024-08-20T22:34:28.6381357Z pytest-rerunfailures==14.0 2024-08-20T22:34:28.6381756Z pytest-xdist==3.3.1 2024-08-20T22:34:28.6382184Z python-dateutil==2.9.0.post0 2024-08-20T22:34:28.6383102Z python-doctr @ git+https://github.com/xuzhao9/doctr.git@7f8a3e61bd6480c4e7b7cde47af0aa2153309c56 2024-08-20T22:34:28.6384514Z pytorch-labs-segment-anything-fast @ git+https://github.com/pytorch-labs/segment-anything-fast.git@de861af9badd03e8b40f5e063d70754a3dc6b4f4 2024-08-20T22:34:28.6385482Z pytz==2024.1 2024-08-20T22:34:28.6385779Z PyWavelets==1.4.1 2024-08-20T22:34:28.6386167Z PyYAML @ file:///croot/pyyaml_1698096049011/work 2024-08-20T22:34:28.6386615Z rapidfuzz==3.9.6 2024-08-20T22:34:28.6386915Z rdflib==7.0.0 2024-08-20T22:34:28.6387204Z redis==5.0.8 2024-08-20T22:34:28.6387513Z referencing==0.35.1 2024-08-20T22:34:28.6387822Z regex==2024.7.24 2024-08-20T22:34:28.6388119Z requests==2.32.3 2024-08-20T22:34:28.6388484Z requests-oauthlib==2.0.0 2024-08-20T22:34:28.6388821Z resampy==0.4.3 2024-08-20T22:34:28.6389118Z rfc3986==1.5.0 2024-08-20T22:34:28.6389406Z rich==13.7.1 2024-08-20T22:34:28.6389692Z rockset==1.0.3 2024-08-20T22:34:28.6390015Z rpds-py==0.20.0 2024-08-20T22:34:28.6390313Z rsa==4.9 2024-08-20T22:34:28.6390591Z s3transfer==0.5.2 2024-08-20T22:34:28.6390898Z safetensors==0.4.4 2024-08-20T22:34:28.6391255Z scikit-image==0.21.0 2024-08-20T22:34:28.6391618Z scikit-learn==1.5.1 2024-08-20T22:34:28.6391938Z scipy==1.10.1 2024-08-20T22:34:28.6392228Z segments==2.2.1 2024-08-20T22:34:28.6392538Z sentencepiece==0.2.0 2024-08-20T22:34:28.6392895Z sentry-sdk==2.13.0 2024-08-20T22:34:28.6393205Z setproctitle==1.3.3 2024-08-20T22:34:28.6393531Z shapely==2.0.6 2024-08-20T22:34:28.6393830Z shellingham==1.5.4 2024-08-20T22:34:28.6394142Z simplejson==3.19.3 2024-08-20T22:34:28.6394537Z six @ file:///tmp/build/80754af9/six_1644875935023/work 2024-08-20T22:34:28.6395049Z smart-open==7.0.4 2024-08-20T22:34:28.6395358Z smmap==5.0.1 2024-08-20T22:34:28.6395651Z sortedcontainers==2.4.0 2024-08-20T22:34:28.6395991Z soundfile==0.12.1 2024-08-20T22:34:28.6396299Z soupsieve==2.6 2024-08-20T22:34:28.6396590Z soxr==0.4.0 2024-08-20T22:34:28.6396862Z spacy==3.7.6 2024-08-20T22:34:28.6397194Z spacy-legacy==3.0.12 2024-08-20T22:34:28.6397560Z spacy-loggers==1.0.5 2024-08-20T22:34:28.6397878Z srsly==2.4.8 2024-08-20T22:34:28.6398159Z stempeg==0.2.3 2024-08-20T22:34:28.6398455Z submitit==1.5.1 2024-08-20T22:34:28.6398749Z sympy==1.13.1 2024-08-20T22:34:28.6399030Z tabulate==0.9.0 2024-08-20T22:34:28.6399380Z tb-nightly==2.13.0a20230426 2024-08-20T22:34:28.6399737Z tensorboard==2.13.0 2024-08-20T22:34:28.6400117Z tensorboard-data-server==0.7.2 2024-08-20T22:34:28.6400498Z tensorboardX==2.6.2.2 2024-08-20T22:34:28.6400831Z termcolor==2.4.0 2024-08-20T22:34:28.6401131Z thinc==8.2.5 2024-08-20T22:34:28.6401424Z threadpoolctl==3.5.0 2024-08-20T22:34:28.6401749Z tifffile==2024.8.10 2024-08-20T22:34:28.6402555Z timm @ git+https://github.com/huggingface/pytorch-image-models.git@730b907b4d45a4713cbc425cbf224c46089fd514 2024-08-20T22:34:28.6403338Z tlparse==0.3.25 2024-08-20T22:34:28.6403637Z tokenizers==0.15.2 2024-08-20T22:34:28.6403941Z tomli==2.0.1 2024-08-20T22:34:28.6404227Z tomlkit==0.13.2 2024-08-20T22:34:28.6405427Z torch @ file:///var/lib/jenkins/workspace/dist/torch-2.5.0a0%2Bgit40ec5f6-cp310-cp310-linux_x86_64.whl#sha256=b70a70c0c1fc7cc3f43c6342d758f4ffd3a540a88b786d266604d87618be5127 2024-08-20T22:34:28.6407041Z torch_geometric @ git+https://github.com/pyg-team/pytorch_geometric.git@cabcd4097442ba60aa1efa11e1619dd9bb8fb527 2024-08-20T22:34:28.6408099Z torchao @ git+https://github.com/pytorch/ao.git@2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T22:34:28.6409204Z torchaudio @ git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:34:28.6410226Z torchdata @ git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:34:28.6411413Z torchmultimodal @ git+https://github.com/facebookresearch/multimodal.git@e4d288b45b89cee462a21ab264405f3f368adc21 2024-08-20T22:34:28.6412491Z torchtext @ git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:34:28.6413483Z torchvision @ git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:34:28.6414179Z tornado==6.4.1 2024-08-20T22:34:28.6414479Z tqdm==4.66.5 2024-08-20T22:34:28.6414773Z transformers==4.38.1 2024-08-20T22:34:28.6415105Z treetable==0.2.5 2024-08-20T22:34:28.6415922Z -e git+https://github.com/openai/triton@dedb7bdf339a3546896d4820366ca562c586bfa0#egg=triton&subdirectory=python 2024-08-20T22:34:28.6416710Z typeguard==4.3.0 2024-08-20T22:34:28.6417023Z typer==0.12.4 2024-08-20T22:34:28.6417334Z typing_extensions==4.12.2 2024-08-20T22:34:28.6417678Z tzdata==2024.1 2024-08-20T22:34:28.6417969Z Unidecode==1.3.8 2024-08-20T22:34:28.6418357Z unittest-xml-reporting==3.2.0 2024-08-20T22:34:28.6418737Z uritemplate==4.1.1 2024-08-20T22:34:28.6419051Z urllib3==1.26.19 2024-08-20T22:34:28.6419349Z visdom==0.2.4 2024-08-20T22:34:28.6419644Z wandb==0.17.7 2024-08-20T22:34:28.6419937Z wasabi==1.1.3 2024-08-20T22:34:28.6420229Z wcwidth==0.2.13 2024-08-20T22:34:28.6420531Z weasel==0.4.1 2024-08-20T22:34:28.6420882Z websocket-client==1.8.0 2024-08-20T22:34:28.6421228Z Werkzeug==3.0.3 2024-08-20T22:34:28.6421525Z wrapt==1.16.0 2024-08-20T22:34:28.6421821Z xdoctest==1.1.0 2024-08-20T22:34:28.6422118Z xxhash==3.5.0 2024-08-20T22:34:28.6422412Z xyzservices==2024.6.0 2024-08-20T22:34:28.6422835Z yacs==0.1.8 2024-08-20T22:34:28.6423116Z yarl==1.9.4 2024-08-20T22:34:28.6423446Z z3-solver==4.12.2.0 2024-08-20T22:34:28.6423755Z zipp==3.20.0 2024-08-20T22:34:28.6616254Z + popd 2024-08-20T22:34:28.6616574Z ~/workspace 2024-08-20T22:34:28.6617634Z + [[ dynamo_eager_torchbench != *cpu* ]] 2024-08-20T22:34:28.6618292Z + install_torchrec_and_fbgemm 2024-08-20T22:34:28.6618722Z + local torchrec_commit 2024-08-20T22:34:28.6621145Z ++ get_pinned_commit torchrec 2024-08-20T22:34:28.6621705Z ++ cat .github/ci_commit_pins/torchrec.txt 2024-08-20T22:34:28.6640266Z + torchrec_commit=6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T22:34:28.6640992Z + local fbgemm_commit 2024-08-20T22:34:28.6644119Z ++ get_pinned_commit fbgemm 2024-08-20T22:34:28.6644657Z ++ cat .github/ci_commit_pins/fbgemm.txt 2024-08-20T22:34:28.6662684Z + fbgemm_commit=de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:34:28.6663562Z + pip_uninstall torchrec-nightly 2024-08-20T22:34:28.6664040Z + pip uninstall -y torchrec-nightly 2024-08-20T22:34:29.0425995Z WARNING: Skipping torchrec-nightly as it is not installed. 2024-08-20T22:34:29.0775958Z + pip_uninstall fbgemm-gpu-nightly 2024-08-20T22:34:29.0777062Z + pip uninstall -y fbgemm-gpu-nightly 2024-08-20T22:34:29.4459081Z WARNING: Skipping fbgemm-gpu-nightly as it is not installed. 2024-08-20T22:34:29.4819572Z + pip_install setuptools-git-versioning scikit-build pyre-extensions 2024-08-20T22:34:29.4820574Z + pip install --progress-bar off setuptools-git-versioning scikit-build pyre-extensions 2024-08-20T22:34:29.9703728Z Collecting setuptools-git-versioning 2024-08-20T22:34:29.9859741Z Downloading setuptools_git_versioning-2.0.0-py3-none-any.whl.metadata (5.8 kB) 2024-08-20T22:34:30.1079858Z Collecting scikit-build 2024-08-20T22:34:30.1117964Z Downloading scikit_build-0.18.0-py3-none-any.whl.metadata (17 kB) 2024-08-20T22:34:30.2349774Z Collecting pyre-extensions 2024-08-20T22:34:30.2380264Z Downloading pyre_extensions-0.0.30-py3-none-any.whl.metadata (4.0 kB) 2024-08-20T22:34:30.2926786Z Requirement already satisfied: packaging in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from setuptools-git-versioning) (24.1) 2024-08-20T22:34:30.2929757Z Requirement already satisfied: setuptools in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from setuptools-git-versioning) (72.1.0) 2024-08-20T22:34:30.3344751Z Collecting toml>=0.10.2 (from setuptools-git-versioning) 2024-08-20T22:34:30.3371744Z Downloading toml-0.10.2-py2.py3-none-any.whl.metadata (7.1 kB) 2024-08-20T22:34:30.3895862Z Requirement already satisfied: distro in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (1.9.0) 2024-08-20T22:34:30.3900935Z Requirement already satisfied: tomli in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (2.0.1) 2024-08-20T22:34:30.3904790Z Requirement already satisfied: wheel>=0.32.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from scikit-build) (0.43.0) 2024-08-20T22:34:30.4533589Z Collecting typing-inspect (from pyre-extensions) 2024-08-20T22:34:30.4564411Z Downloading typing_inspect-0.9.0-py3-none-any.whl.metadata (1.5 kB) 2024-08-20T22:34:30.5075516Z Requirement already satisfied: typing-extensions in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pyre-extensions) (4.12.2) 2024-08-20T22:34:30.5150957Z Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from typing-inspect->pyre-extensions) (1.0.0) 2024-08-20T22:34:30.5183945Z Downloading setuptools_git_versioning-2.0.0-py3-none-any.whl (10 kB) 2024-08-20T22:34:30.5731198Z Downloading scikit_build-0.18.0-py3-none-any.whl (85 kB) 2024-08-20T22:34:30.6329200Z Downloading pyre_extensions-0.0.30-py3-none-any.whl (12 kB) 2024-08-20T22:34:30.6843974Z Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-08-20T22:34:30.7405862Z Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB) 2024-08-20T22:34:31.6205063Z Installing collected packages: typing-inspect, toml, scikit-build, setuptools-git-versioning, pyre-extensions 2024-08-20T22:34:31.9822609Z Successfully installed pyre-extensions-0.0.30 scikit-build-0.18.0 setuptools-git-versioning-2.0.0 toml-0.10.2 typing-inspect-0.9.0 2024-08-20T22:34:32.0952461Z + CUDA_PATH=/usr/local/cuda-12.1 2024-08-20T22:34:32.0953669Z + pip_install --no-use-pep517 --user 'git+https://github.com/pytorch/FBGEMM.git@de731af65b4f04696e85c729e3282450b51b95fd#egg=fbgemm-gpu&subdirectory=fbgemm_gpu' 2024-08-20T22:34:32.0955535Z + pip install --progress-bar off --no-use-pep517 --user 'git+https://github.com/pytorch/FBGEMM.git@de731af65b4f04696e85c729e3282450b51b95fd#egg=fbgemm-gpu&subdirectory=fbgemm_gpu' 2024-08-20T22:34:32.4645466Z Collecting fbgemm-gpu 2024-08-20T22:34:32.4648335Z Cloning https://github.com/pytorch/FBGEMM.git (to revision de731af65b4f04696e85c729e3282450b51b95fd) to /tmp/pip-install-aqt1vnlo/fbgemm-gpu_52342a2eea034f2e96c589bb2850876b 2024-08-20T22:34:32.4679005Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/FBGEMM.git /tmp/pip-install-aqt1vnlo/fbgemm-gpu_52342a2eea034f2e96c589bb2850876b 2024-08-20T22:34:33.6144550Z Running command git rev-parse -q --verify 'sha^de731af65b4f04696e85c729e3282450b51b95fd' 2024-08-20T22:34:33.6171899Z Running command git fetch -q https://github.com/pytorch/FBGEMM.git de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:34:34.1130916Z Running command git checkout -q de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:34:34.6622825Z Resolved https://github.com/pytorch/FBGEMM.git to commit de731af65b4f04696e85c729e3282450b51b95fd 2024-08-20T22:34:34.6623970Z Running command git submodule update --init --recursive -q 2024-08-20T22:34:44.0650260Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T22:34:44.0670386Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbgemm-gpu) (1.21.2) 2024-08-20T22:34:44.0678477Z Building wheels for collected packages: fbgemm-gpu 2024-08-20T23:19:22.6605173Z Building wheel for fbgemm-gpu (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done 2024-08-20T23:19:22.8443581Z [?25h Created wheel for fbgemm-gpu: filename=fbgemm_gpu-0.4.1rc0.post421-cp310-cp310-linux_x86_64.whl size=252620346 sha256=cfd5b16ea94961e53543fb9ffd58873f685fdf3aded227662efd5dfd033a7c3b 2024-08-20T23:19:22.8445245Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/d8/d9/f3/f5c260aba9c9f0a533444c12621a0c2ceb8fbcffce2b5beb39 2024-08-20T23:19:22.8464718Z Successfully built fbgemm-gpu 2024-08-20T23:19:23.6830577Z Installing collected packages: fbgemm-gpu 2024-08-20T23:19:27.0199759Z Successfully installed fbgemm-gpu-0.4.1rc0.post421 2024-08-20T23:19:27.9459887Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:27.9461445Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:28.3172204Z Collecting git+https://github.com/pytorch/torchrec.git@6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:28.3175973Z Cloning https://github.com/pytorch/torchrec.git (to revision 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2) to /tmp/pip-req-build-ipx62yv8 2024-08-20T23:19:28.3206004Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/torchrec.git /tmp/pip-req-build-ipx62yv8 2024-08-20T23:19:29.0779927Z Running command git rev-parse -q --verify 'sha^6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2' 2024-08-20T23:19:29.0806513Z Running command git fetch -q https://github.com/pytorch/torchrec.git 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:29.3844040Z Running command git checkout -q 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:29.7323466Z Resolved https://github.com/pytorch/torchrec.git to commit 6cd9fd362514d14ebb9ed51314c62ac1e1e2bbf2 2024-08-20T23:19:30.0066561Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T23:19:30.0101409Z [?25hRequirement already satisfied: iopath in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (0.1.9) 2024-08-20T23:19:30.0105422Z Requirement already satisfied: pandas in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (2.0.3) 2024-08-20T23:19:30.0109052Z Requirement already satisfied: tabulate in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (0.9.0) 2024-08-20T23:19:30.0572096Z Collecting torchmetrics (from torchrec==0.3.2) 2024-08-20T23:19:30.0745727Z Downloading torchmetrics-1.4.1-py3-none-any.whl.metadata (20 kB) 2024-08-20T23:19:30.1123632Z Requirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchrec==0.3.2) (4.66.5) 2024-08-20T23:19:30.1125690Z Requirement already satisfied: fbgemm-gpu in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchrec==0.3.2) (0.4.1rc0.post421) 2024-08-20T23:19:30.1135448Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbgemm-gpu->torchrec==0.3.2) (1.21.2) 2024-08-20T23:19:30.1145823Z Requirement already satisfied: portalocker in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from iopath->torchrec==0.3.2) (2.10.1) 2024-08-20T23:19:30.1415031Z Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2.9.0.post0) 2024-08-20T23:19:30.1418884Z Requirement already satisfied: pytz>=2020.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2024.1) 2024-08-20T23:19:30.1422016Z Requirement already satisfied: tzdata>=2022.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from pandas->torchrec==0.3.2) (2024.1) 2024-08-20T23:19:30.1443236Z Requirement already satisfied: packaging>17.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchmetrics->torchrec==0.3.2) (24.1) 2024-08-20T23:19:30.1447174Z Requirement already satisfied: torch>=1.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchmetrics->torchrec==0.3.2) (2.5.0a0+git40ec5f6) 2024-08-20T23:19:30.1589539Z Collecting lightning-utilities>=0.8.0 (from torchmetrics->torchrec==0.3.2) 2024-08-20T23:19:30.1624229Z Downloading lightning_utilities-0.11.6-py3-none-any.whl.metadata (5.2 kB) 2024-08-20T23:19:30.1706605Z Requirement already satisfied: setuptools in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from lightning-utilities>=0.8.0->torchmetrics->torchrec==0.3.2) (72.1.0) 2024-08-20T23:19:30.1709451Z Requirement already satisfied: typing-extensions in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from lightning-utilities>=0.8.0->torchmetrics->torchrec==0.3.2) (4.12.2) 2024-08-20T23:19:30.1729936Z Requirement already satisfied: six>=1.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->torchrec==0.3.2) (1.16.0) 2024-08-20T23:19:30.1763451Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (3.13.1) 2024-08-20T23:19:30.1766918Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2.8.8) 2024-08-20T23:19:30.1770471Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (3.1.4) 2024-08-20T23:19:30.1773125Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2024.6.1) 2024-08-20T23:19:30.1778340Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>=1.10.0->torchmetrics->torchrec==0.3.2) (1.13.1) 2024-08-20T23:19:30.1793671Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch>=1.10.0->torchmetrics->torchrec==0.3.2) (1.3.0) 2024-08-20T23:19:30.2199155Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->torchmetrics->torchrec==0.3.2) (2.1.5) 2024-08-20T23:19:30.2332621Z Downloading torchmetrics-1.4.1-py3-none-any.whl (866 kB) 2024-08-20T23:19:30.2511601Z Downloading lightning_utilities-0.11.6-py3-none-any.whl (26 kB) 2024-08-20T23:19:30.2599330Z Building wheels for collected packages: torchrec 2024-08-20T23:19:30.7768522Z Building wheel for torchrec (setup.py) ... [?25l- \ | done 2024-08-20T23:19:30.7774882Z [?25h Created wheel for torchrec: filename=torchrec-0.3.2-py3-none-any.whl size=374488 sha256=ee63a171838d8e1849e1d0d2682e10159a2c4fbf7ff2a1e1047c66529c7b1001 2024-08-20T23:19:30.7776461Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/8e/a1/47/39ede01672ba82c08fb521bfc057cc4347e4b0e951738c8ca8 2024-08-20T23:19:30.7800677Z Successfully built torchrec 2024-08-20T23:19:31.5573072Z Installing collected packages: lightning-utilities, torchmetrics, torchrec 2024-08-20T23:19:32.1662297Z Successfully installed lightning-utilities-0.11.6 torchmetrics-1.4.1 torchrec-0.3.2 2024-08-20T23:19:32.3076434Z ++ pwd 2024-08-20T23:19:32.3079173Z + PYTHONPATH=/var/lib/jenkins/workspace/torchbench 2024-08-20T23:19:32.3079705Z + test_dynamo_benchmark torchbench 0 2024-08-20T23:19:32.3084698Z ++ pwd 2024-08-20T23:19:32.3088731Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:19:32.3089325Z + local suite=torchbench 2024-08-20T23:19:32.3089677Z + shift 2024-08-20T23:19:32.3089960Z + local shard_id=0 2024-08-20T23:19:32.3092674Z + shift 2024-08-20T23:19:32.3093094Z + [[ dynamo_eager_torchbench == *perf_compare* ]] 2024-08-20T23:19:32.3093632Z + [[ dynamo_eager_torchbench == *perf* ]] 2024-08-20T23:19:32.3094120Z + [[ dynamo_eager_torchbench == *cpu* ]] 2024-08-20T23:19:32.3094753Z + [[ dynamo_eager_torchbench == *aot_inductor* ]] 2024-08-20T23:19:32.3095585Z + test_single_dynamo_benchmark inference torchbench 0 --inference --bfloat16 2024-08-20T23:19:32.3096164Z ++ pwd 2024-08-20T23:19:32.3096645Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:19:32.3097325Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T23:19:32.3140075Z + local name=inference 2024-08-20T23:19:32.3140439Z + shift 2024-08-20T23:19:32.3140734Z + local suite=torchbench 2024-08-20T23:19:32.3141084Z + shift 2024-08-20T23:19:32.3141361Z + local shard_id=0 2024-08-20T23:19:32.3141689Z + shift 2024-08-20T23:19:32.3141967Z + partition_flags=() 2024-08-20T23:19:32.3142314Z + local partition_flags 2024-08-20T23:19:32.3142754Z + [[ -n 2 ]] 2024-08-20T23:19:32.3143158Z + [[ -n 0 ]] 2024-08-20T23:19:32.3143785Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T23:19:32.3144488Z + [[ dynamo_eager_torchbench == *perf_compare* ]] 2024-08-20T23:19:32.3144983Z + [[ dynamo_eager_torchbench == *perf* ]] 2024-08-20T23:19:32.3145471Z + [[ dynamo_eager_torchbench == *aot_inductor* ]] 2024-08-20T23:19:32.3145961Z + [[ dynamo_eager_torchbench == *_avx2* ]] 2024-08-20T23:19:32.3146435Z + [[ dynamo_eager_torchbench == *_avx512* ]] 2024-08-20T23:19:32.3148068Z + python benchmarks/dynamo/torchbench.py --ci --accuracy --timing --explain --backend eager --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv 2024-08-20T23:19:38.4857757Z 2024-08-20T23:19:39.2068290Z loading model: 0it [00:00, ?it/s]/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/optim/apply_optimizer_in_backward.py:47: DeprecationWarning: `TorchScript` support for functional optimizers is deprecated and will be removed in a future PyTorch release. Consider using the `torch.compile` optimizer instead. 2024-08-20T23:19:39.2070362Z from torch.distributed.optim import _apply_optimizer_in_backward 2024-08-20T23:19:39.6115720Z 2024-08-20T23:19:39.6116297Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:19:39.6116815Z cuda eval torchrec_dlrm 2024-08-20T23:19:39.6147966Z Traceback (most recent call last): 2024-08-20T23:19:39.6148825Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2507, in validate_model 2024-08-20T23:19:39.6149599Z self.model_iter_fn(model, example_inputs) 2024-08-20T23:19:39.6150358Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 434, in forward_pass 2024-08-20T23:19:39.6151075Z return mod(*inputs) 2024-08-20T23:19:39.6152299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:19:39.6153212Z return self._call_impl(*args, **kwargs) 2024-08-20T23:19:39.6154462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:19:39.6155363Z return forward_call(*args, **kwargs) 2024-08-20T23:19:39.6156437Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/models/dlrm.py", line 893, in forward 2024-08-20T23:19:39.6157584Z logits = self.model(batch.dense_features, batch.sparse_features) 2024-08-20T23:19:39.6158814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:19:39.6159950Z return self._call_impl(*args, **kwargs) 2024-08-20T23:19:39.6161249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:19:39.6162233Z return forward_call(*args, **kwargs) 2024-08-20T23:19:39.6163127Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/models/dlrm.py", line 572, in forward 2024-08-20T23:19:39.6163946Z concatenated_dense = self.inter_arch( 2024-08-20T23:19:39.6164935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:19:39.6165829Z return self._call_impl(*args, **kwargs) 2024-08-20T23:19:39.6166766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:19:39.6167594Z return forward_call(*args, **kwargs) 2024-08-20T23:19:39.6168463Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/models/dlrm.py", line 284, in forward 2024-08-20T23:19:39.6169393Z return self.crossnet(combined_values.reshape([B, -1])) 2024-08-20T23:19:39.6170451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:19:39.6171334Z return self._call_impl(*args, **kwargs) 2024-08-20T23:19:39.6172265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:19:39.6173097Z return forward_call(*args, **kwargs) 2024-08-20T23:19:39.6174010Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchrec/modules/crossnet.py", line 175, in forward 2024-08-20T23:19:39.6175003Z x_l_v = torch.nn.functional.linear(x_l, self.V_kernels[layer]) 2024-08-20T23:19:39.6175837Z RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != c10::BFloat16 2024-08-20T23:19:39.6176399Z 2024-08-20T23:19:39.6176700Z The above exception was the direct cause of the following exception: 2024-08-20T23:19:39.6177149Z 2024-08-20T23:19:39.6177298Z Traceback (most recent call last): 2024-08-20T23:19:39.6177955Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:19:39.6178615Z ) = runner.load_model( 2024-08-20T23:19:39.6179279Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 362, in load_model 2024-08-20T23:19:39.6180046Z self.validate_model(model, example_inputs) 2024-08-20T23:19:39.6180807Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2509, in validate_model 2024-08-20T23:19:39.6181575Z raise RuntimeError("Eager run failed") from e 2024-08-20T23:19:39.6182051Z RuntimeError: Eager run failed 2024-08-20T23:19:39.6182318Z 2024-08-20T23:19:39.6182442Z eager_fail_to_run 2024-08-20T23:19:43.1726382Z 2024-08-20T23:19:45.3282725Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:19:45.3283261Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:19:45.3343536Z cuda eval BERT_pytorch 2024-08-20T23:19:50.2312882Z pass 2024-08-20T23:19:50.2316446Z TIMING: backend_compile:0.00057 entire_frame_compile:4.04391 total_wall_time:4.04391 2024-08-20T23:19:50.2317416Z STATS: call_* op count: 1076 | FakeTensor.__torch_dispatch__:4 | FakeTensorMode.__torch_dispatch__:2999 2024-08-20T23:19:50.2318316Z Dynamo produced 1 graphs covering 1076 ops with 0 graph breaks (0 unique) 2024-08-20T23:19:53.9105751Z 2024-08-20T23:19:56.8504668Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:19:56.8505224Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:19:56.8551087Z cuda eval Background_Matting 2024-08-20T23:19:56.8553966Z pass_due_to_skip 2024-08-20T23:19:56.9085874Z TIMING: 2024-08-20T23:19:56.9086287Z STATS: call_* op count: 0 2024-08-20T23:19:56.9086990Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:20:00.5082007Z 2024-08-20T23:20:03.7818940Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:20:03.7819446Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:20:03.7843278Z cuda eval LearningToPaint 2024-08-20T23:20:06.0904822Z pass 2024-08-20T23:20:06.0907673Z TIMING: backend_compile:0.00026 entire_frame_compile:1.79816 total_wall_time:1.79816 2024-08-20T23:20:06.0908541Z STATS: call_* op count: 142 | FakeTensorMode.__torch_dispatch__:1430 2024-08-20T23:20:06.0909288Z Dynamo produced 1 graphs covering 142 ops with 0 graph breaks (0 unique) 2024-08-20T23:20:09.9618301Z 2024-08-20T23:20:11.3870867Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/vgg16-397923af.pth 2024-08-20T23:20:11.4037266Z 2024-08-20T23:20:11.4037483Z 2024-08-20T23:20:11.5042643Z 0% 0.00/528M [00:00 will be ignored 2024-08-20T23:34:28.6422350Z pass 2024-08-20T23:34:28.6898909Z TIMING: entire_frame_compile:9.55837 backend_compile:0.00119 total_wall_time:9.55837 2024-08-20T23:34:28.6899892Z STATS: call_* op count: 671 | FakeTensorMode.__torch_dispatch__:5295 | FakeTensor.__torch_dispatch__:1668 2024-08-20T23:34:28.6900792Z Dynamo produced 3 graphs covering 671 ops with 7 graph breaks (5 unique) 2024-08-20T23:34:32.5637744Z 2024-08-20T23:34:34.0590092Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:34:34.0590695Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:34:34.0593467Z cuda train alexnet 2024-08-20T23:34:35.9712319Z pass 2024-08-20T23:34:35.9919310Z TIMING: entire_frame_compile:0.5496 backend_compile:0.00015 total_wall_time:0.5496 2024-08-20T23:34:35.9920515Z STATS: call_* op count: 26 | FakeTensorMode.__torch_dispatch__:414 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:34:35.9921443Z Dynamo produced 2 graphs covering 26 ops with 6 graph breaks (5 unique) 2024-08-20T23:34:39.6242840Z 2024-08-20T23:34:41.5451969Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:34:41.5452574Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:34:41.5453095Z cuda train basic_gnn_edgecnn 2024-08-20T23:34:43.7826327Z W0820 23:34:43.781000 35677 torch/_logging/_internal.py:1058] [24/0] Profiler function will be ignored 2024-08-20T23:34:44.3518811Z pass 2024-08-20T23:34:44.3690542Z TIMING: entire_frame_compile:1.84384 backend_compile:0.00057 total_wall_time:1.84384 2024-08-20T23:34:44.3691568Z STATS: call_* op count: 102 | FakeTensorMode.__torch_dispatch__:919 | FakeTensor.__torch_dispatch__:376 2024-08-20T23:34:44.3692467Z Dynamo produced 8 graphs covering 102 ops with 20 graph breaks (6 unique) 2024-08-20T23:34:48.1577216Z 2024-08-20T23:34:50.1347612Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:34:50.1348670Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:34:50.1349143Z cuda train basic_gnn_gcn 2024-08-20T23:34:51.3534046Z W0820 23:34:51.352000 35767 torch/_logging/_internal.py:1058] [18/0] Profiler function will be ignored 2024-08-20T23:34:51.7051675Z pass 2024-08-20T23:34:51.7164595Z TIMING: entire_frame_compile:0.9683 backend_compile:0.0006 total_wall_time:0.9683 2024-08-20T23:34:51.7165647Z STATS: call_* op count: 98 | FakeTensorMode.__torch_dispatch__:392 | FakeTensor.__torch_dispatch__:176 2024-08-20T23:34:51.7166548Z Dynamo produced 10 graphs covering 98 ops with 13 graph breaks (6 unique) 2024-08-20T23:34:55.4742786Z 2024-08-20T23:34:57.4040982Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:34:57.4041691Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:34:57.4042358Z cuda train basic_gnn_gin 2024-08-20T23:34:58.6259472Z W0820 23:34:58.625000 35857 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:34:59.1852959Z pass 2024-08-20T23:34:59.1940687Z TIMING: entire_frame_compile:1.20068 backend_compile:0.0003 total_wall_time:1.20068 2024-08-20T23:34:59.1941708Z STATS: call_* op count: 109 | FakeTensorMode.__torch_dispatch__:499 | FakeTensor.__torch_dispatch__:236 2024-08-20T23:34:59.1942699Z Dynamo produced 3 graphs covering 109 ops with 7 graph breaks (5 unique) 2024-08-20T23:35:02.9780718Z 2024-08-20T23:35:04.8833651Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:35:04.8834208Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:35:04.8834659Z cuda train basic_gnn_sage 2024-08-20T23:35:05.9747636Z W0820 23:35:05.973000 35947 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:35:06.4513443Z pass 2024-08-20T23:35:06.4578749Z TIMING: entire_frame_compile:0.99992 backend_compile:0.00027 total_wall_time:0.99992 2024-08-20T23:35:06.4579787Z STATS: call_* op count: 106 | FakeTensorMode.__torch_dispatch__:459 | FakeTensor.__torch_dispatch__:182 2024-08-20T23:35:06.4580695Z Dynamo produced 3 graphs covering 106 ops with 7 graph breaks (5 unique) 2024-08-20T23:35:10.2390061Z 2024-08-20T23:35:10.8229735Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:35:10.8230275Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:35:10.8230715Z cuda train dcgan 2024-08-20T23:35:11.6665965Z pass 2024-08-20T23:35:11.6683852Z TIMING: entire_frame_compile:0.54894 backend_compile:0.00014 total_wall_time:0.54894 2024-08-20T23:35:11.6684809Z STATS: call_* op count: 17 | FakeTensorMode.__torch_dispatch__:376 | FakeTensor.__torch_dispatch__:28 2024-08-20T23:35:11.6685694Z Dynamo produced 2 graphs covering 17 ops with 6 graph breaks (5 unique) 2024-08-20T23:35:15.2932556Z 2024-08-20T23:35:18.9317927Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:35:18.9318515Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:35:18.9319072Z cuda train demucs 2024-08-20T23:36:15.3868256Z pass 2024-08-20T23:36:15.4097473Z TIMING: entire_frame_compile:1.77304 backend_compile:0.00042 total_wall_time:1.77304 2024-08-20T23:36:15.4098697Z STATS: call_* op count: 88 | FakeTensorMode.__torch_dispatch__:1218 | FakeTensor.__torch_dispatch__:47 2024-08-20T23:36:15.4099641Z Dynamo produced 6 graphs covering 88 ops with 9 graph breaks (6 unique) 2024-08-20T23:36:19.0767917Z 2024-08-20T23:36:21.3596292Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:21.3596787Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:36:21.3597244Z cuda train densenet121 2024-08-20T23:36:42.4897331Z pass 2024-08-20T23:36:42.5021524Z TIMING: entire_frame_compile:15.46164 backend_compile:0.00125 total_wall_time:15.46164 2024-08-20T23:36:42.5022814Z STATS: call_* op count: 435 | FakeTensorMode.__torch_dispatch__:11864 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:36:42.5024326Z Dynamo produced 2 graphs covering 435 ops with 6 graph breaks (5 unique) 2024-08-20T23:36:46.4313566Z 2024-08-20T23:36:51.4734835Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:36:51.4735446Z loading model: 0it [00:05, ?it/s] 2024-08-20T23:36:51.4736262Z cuda train detectron2_maskrcnn_r_50_c4 2024-08-20T23:36:51.4749835Z Traceback (most recent call last): 2024-08-20T23:36:51.4750733Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2507, in validate_model 2024-08-20T23:36:51.4751515Z self.model_iter_fn(model, example_inputs) 2024-08-20T23:36:51.4752556Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 443, in forward_and_backward_pass 2024-08-20T23:36:51.4753545Z pred = mod(*cloned_inputs) 2024-08-20T23:36:51.4754922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:36:51.4755839Z return self._call_impl(*args, **kwargs) 2024-08-20T23:36:51.4756811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:36:51.4757647Z return forward_call(*args, **kwargs) 2024-08-20T23:36:51.4758772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T23:36:51.4759815Z return self.inference(batched_inputs) 2024-08-20T23:36:51.4760921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 213, in inference 2024-08-20T23:36:51.4762109Z results, _ = self.roi_heads(images, features, proposals, None) 2024-08-20T23:36:51.4763317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:36:51.4764421Z return self._call_impl(*args, **kwargs) 2024-08-20T23:36:51.4765656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:36:51.4766604Z return forward_call(*args, **kwargs) 2024-08-20T23:36:51.4767621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/detectron2/modeling/roi_heads/roi_heads.py", line 477, in forward 2024-08-20T23:36:51.4768553Z box_features = self._shared_roi_transform( 2024-08-20T23:36:51.4769706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/detectron2/modeling/roi_heads/roi_heads.py", line 456, in _shared_roi_transform 2024-08-20T23:36:51.4770680Z x = self.pooler(features, boxes) 2024-08-20T23:36:51.4771644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:36:51.4772542Z return self._call_impl(*args, **kwargs) 2024-08-20T23:36:51.4773486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:36:51.4774322Z return forward_call(*args, **kwargs) 2024-08-20T23:36:51.4775247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/detectron2/modeling/poolers.py", line 246, in forward 2024-08-20T23:36:51.4776177Z return self.level_poolers[0](x[0], pooler_fmt_boxes) 2024-08-20T23:36:51.4777230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T23:36:51.4778114Z return self._call_impl(*args, **kwargs) 2024-08-20T23:36:51.4779102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T23:36:51.4779924Z return forward_call(*args, **kwargs) 2024-08-20T23:36:51.4780831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/detectron2/layers/roi_align.py", line 58, in forward 2024-08-20T23:36:51.4781624Z return roi_align( 2024-08-20T23:36:51.4782455Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/roi_align.py", line 236, in roi_align 2024-08-20T23:36:51.4783914Z return _roi_align(input, rois, spatial_scale, output_size[0], output_size[1], sampling_ratio, aligned) 2024-08-20T23:36:51.4785141Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/roi_align.py", line 168, in _roi_align 2024-08-20T23:36:51.4786334Z val = _bilinear_interpolate(input, roi_batch_ind, y, x, ymask, xmask) # [K, C, PH, PW, IY, IX] 2024-08-20T23:36:51.4787575Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/roi_align.py", line 62, in _bilinear_interpolate 2024-08-20T23:36:51.4788448Z v1 = masked_index(y_low, x_low) 2024-08-20T23:36:51.4789352Z File "/var/lib/jenkins/.local/lib/python3.10/site-packages/torchvision/ops/roi_align.py", line 55, in masked_index 2024-08-20T23:36:51.4790172Z return input[ 2024-08-20T23:36:51.4793283Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 21890.87 GiB. GPU 0 has a total capacity of 21.98 GiB of which 16.54 GiB is free. Process 331133 has 5.42 GiB memory in use. Of the allocated memory 4.58 GiB is allocated by PyTorch, and 558.89 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 2024-08-20T23:36:51.4795941Z 2024-08-20T23:36:51.4796241Z The above exception was the direct cause of the following exception: 2024-08-20T23:36:51.4796693Z 2024-08-20T23:36:51.4796846Z Traceback (most recent call last): 2024-08-20T23:36:51.4797511Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:36:51.4798172Z ) = runner.load_model( 2024-08-20T23:36:51.4798841Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 362, in load_model 2024-08-20T23:36:51.4799615Z self.validate_model(model, example_inputs) 2024-08-20T23:36:51.4800382Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2509, in validate_model 2024-08-20T23:36:51.4801157Z raise RuntimeError("Eager run failed") from e 2024-08-20T23:36:51.4801631Z RuntimeError: Eager run failed 2024-08-20T23:36:51.4801899Z 2024-08-20T23:36:51.4802026Z eager_fail_to_run 2024-08-20T23:36:55.2206884Z 2024-08-20T23:37:04.0589781Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:04.0590487Z loading model: 0it [00:08, ?it/s] 2024-08-20T23:37:04.0591215Z cuda train dlrm 2024-08-20T23:37:05.4593471Z pass 2024-08-20T23:37:05.5004922Z TIMING: entire_frame_compile:0.72546 backend_compile:0.00016 total_wall_time:0.72546 2024-08-20T23:37:05.5006273Z STATS: call_* op count: 40 | FakeTensorMode.__torch_dispatch__:611 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:37:05.5007592Z Dynamo produced 2 graphs covering 40 ops with 6 graph breaks (5 unique) 2024-08-20T23:37:09.1096832Z 2024-08-20T23:37:10.5070278Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:10.5070874Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:37:10.5071434Z cuda train drq 2024-08-20T23:37:11.4284999Z W0820 23:37:11.427000 36330 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:37:12.6666803Z pass 2024-08-20T23:37:12.6688164Z TIMING: entire_frame_compile:1.85024 backend_compile:0.00053 total_wall_time:1.85024 2024-08-20T23:37:12.6689178Z STATS: call_* op count: 136 | FakeTensorMode.__torch_dispatch__:875 | FakeTensor.__torch_dispatch__:343 2024-08-20T23:37:12.6690078Z Dynamo produced 5 graphs covering 136 ops with 7 graph breaks (5 unique) 2024-08-20T23:37:16.4143880Z 2024-08-20T23:37:18.2382066Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:18.2382619Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:37:18.2383071Z cuda train fastNLP_Bert 2024-08-20T23:37:24.0100518Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] Graph break from `Tensor.item()`, consider setting: 2024-08-20T23:37:24.0102294Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] torch._dynamo.config.capture_scalar_outputs = True 2024-08-20T23:37:24.0103708Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] or: 2024-08-20T23:37:24.0105011Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-08-20T23:37:24.0106190Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] to include these operations in the captured graph. 2024-08-20T23:37:24.0107162Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] 2024-08-20T23:37:24.0108126Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] Graph break: from user code at: 2024-08-20T23:37:24.0110133Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 443, in torch_dynamo_resume_in_forward_and_backward_pass_at_438 2024-08-20T23:37:24.0111809Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] pred = mod(*cloned_inputs) 2024-08-20T23:37:24.0113545Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/fastNLP/models/bert.py", line 265, in forward 2024-08-20T23:37:24.0114976Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] sequence_output = self.bert(words) 2024-08-20T23:37:24.0116946Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/fastNLP/embeddings/bert_embedding.py", line 137, in forward 2024-08-20T23:37:24.0118502Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] outputs = self.model(words) 2024-08-20T23:37:24.0120144Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/fastNLP/embeddings/bert_embedding.py", line 445, in forward 2024-08-20T23:37:24.0122165Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] max_word_piece_length = batch_word_pieces_length.sum(dim=-1).max().item() # 表示word piece的长度(包括padding) 2024-08-20T23:37:24.0125158Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] 2024-08-20T23:37:24.0125901Z W0820 23:37:24.009000 36383 torch/_dynamo/variables/tensor.py:776] [3/0] 2024-08-20T23:37:27.9508170Z pass 2024-08-20T23:37:28.0429989Z TIMING: entire_frame_compile:3.73679 backend_compile:0.00081 total_wall_time:3.73679 2024-08-20T23:37:28.0430972Z STATS: call_* op count: 448 | FakeTensorMode.__torch_dispatch__:3653 | FakeTensor.__torch_dispatch__:20 2024-08-20T23:37:28.0431881Z Dynamo produced 7 graphs covering 448 ops with 10 graph breaks (6 unique) 2024-08-20T23:37:31.7470792Z 2024-08-20T23:37:32.6500814Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:32.6501435Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:32.6501955Z cuda train functorch_dp_cifar10 2024-08-20T23:37:34.7141283Z W0820 23:37:34.713000 36427 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:37:37.7701070Z pass 2024-08-20T23:37:37.7764424Z TIMING: entire_frame_compile:4.56685 backend_compile:0.00078 total_wall_time:4.56685 2024-08-20T23:37:37.7766373Z STATS: call_* op count: 276 | FakeTensorMode.__torch_dispatch__:1391 | FakeTensor.__torch_dispatch__:1136 2024-08-20T23:37:37.7768173Z Dynamo produced 3 graphs covering 276 ops with 7 graph breaks (5 unique) 2024-08-20T23:37:41.5394727Z 2024-08-20T23:37:42.1691267Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:42.1691810Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:42.1692262Z cuda train functorch_maml_omniglot 2024-08-20T23:37:42.8972618Z W0820 23:37:42.896000 36471 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:37:43.5327843Z pass 2024-08-20T23:37:43.5329105Z TIMING: entire_frame_compile:1.07546 backend_compile:0.0003 total_wall_time:1.07546 2024-08-20T23:37:43.5330074Z STATS: call_* op count: 77 | FakeTensorMode.__torch_dispatch__:385 | FakeTensor.__torch_dispatch__:272 2024-08-20T23:37:43.5331116Z Dynamo produced 3 graphs covering 77 ops with 7 graph breaks (5 unique) 2024-08-20T23:37:47.1595696Z 2024-08-20T23:37:48.4920045Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:37:48.4920768Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:37:48.4921442Z cuda train hf_Albert 2024-08-20T23:37:57.6125047Z W0820 23:37:57.611000 36515 torch/_logging/_internal.py:1058] [5/0] Profiler function will be ignored 2024-08-20T23:37:58.9200260Z pass 2024-08-20T23:37:58.9934256Z TIMING: entire_frame_compile:3.6679 backend_compile:0.00052 total_wall_time:3.6679 2024-08-20T23:37:58.9935633Z STATS: call_* op count: 541 | FakeTensorMode.__torch_dispatch__:6131 | FakeTensor.__torch_dispatch__:523 2024-08-20T23:37:58.9937338Z Dynamo produced 3 graphs covering 541 ops with 6 graph breaks (5 unique) 2024-08-20T23:38:02.6851096Z 2024-08-20T23:38:06.6746655Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:06.6747158Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:38:06.6748936Z cuda train hf_Bart 2024-08-20T23:38:22.1735459Z W0820 23:38:22.172000 36559 torch/_logging/_internal.py:1058] [5/0] Profiler function will be ignored 2024-08-20T23:38:32.5547313Z pass 2024-08-20T23:38:32.5993049Z TIMING: entire_frame_compile:16.03242 backend_compile:0.00317 total_wall_time:16.03242 2024-08-20T23:38:32.5994172Z STATS: call_* op count: 1249 | FakeTensorMode.__torch_dispatch__:7712 | FakeTensor.__torch_dispatch__:4790 2024-08-20T23:38:32.5995072Z Dynamo produced 3 graphs covering 1249 ops with 6 graph breaks (5 unique) 2024-08-20T23:38:36.6157802Z 2024-08-20T23:38:39.2207015Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:38:39.2208170Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:38:39.2209175Z cuda train hf_Bert 2024-08-20T23:38:52.2570181Z W0820 23:38:52.255000 36603 torch/_logging/_internal.py:1058] [5/0] Profiler function will be ignored 2024-08-20T23:39:00.3995430Z pass 2024-08-20T23:39:00.4384669Z TIMING: entire_frame_compile:13.39916 backend_compile:0.00236 total_wall_time:13.39916 2024-08-20T23:39:00.4385835Z STATS: call_* op count: 994 | FakeTensorMode.__torch_dispatch__:6713 | FakeTensor.__torch_dispatch__:3655 2024-08-20T23:39:00.4386747Z Dynamo produced 3 graphs covering 994 ops with 6 graph breaks (5 unique) 2024-08-20T23:39:04.3693604Z 2024-08-20T23:39:09.1940289Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:39:09.1940807Z loading model: 0it [00:04, ?it/s] 2024-08-20T23:39:09.1943134Z cuda train hf_Bert_large 2024-08-20T23:39:39.1535376Z W0820 23:39:39.152000 36647 torch/_logging/_internal.py:1058] [5/0] Profiler function will be ignored 2024-08-20T23:39:54.9753051Z pass 2024-08-20T23:39:55.1265959Z TIMING: entire_frame_compile:26.03869 backend_compile:0.00512 total_wall_time:26.03869 2024-08-20T23:39:55.1266985Z STATS: call_* op count: 1918 | FakeTensorMode.__torch_dispatch__:13049 | FakeTensor.__torch_dispatch__:7111 2024-08-20T23:39:55.1267915Z Dynamo produced 3 graphs covering 1918 ops with 6 graph breaks (5 unique) 2024-08-20T23:39:59.3745983Z 2024-08-20T23:40:02.9682447Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:40:02.9683182Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:40:02.9683853Z cuda train hf_BigBird 2024-08-20T23:40:14.2493270Z WARNING:common:fp64 golden ref were not generated for hf_BigBird. Setting accuracy check to cosine 2024-08-20T23:40:31.9077747Z W0820 23:40:31.906000 36691 torch/_logging/_internal.py:1058] [5/0] Profiler function will be ignored 2024-08-20T23:40:40.5085391Z pass 2024-08-20T23:40:40.5701627Z TIMING: entire_frame_compile:21.99855 backend_compile:0.00299 total_wall_time:21.99855 2024-08-20T23:40:40.5702802Z STATS: call_* op count: 3974 | FakeTensorMode.__torch_dispatch__:37293 | FakeTensor.__torch_dispatch__:3655 2024-08-20T23:40:40.5703946Z Dynamo produced 3 graphs covering 3974 ops with 6 graph breaks (5 unique) 2024-08-20T23:40:44.6751765Z 2024-08-20T23:40:46.7772352Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:40:46.7772831Z loading model: 0it [00:02, ?it/s] 2024-08-20T23:40:46.7773285Z cuda train hf_DistilBert 2024-08-20T23:40:53.8114330Z W0820 23:40:53.810000 36735 torch/_logging/_internal.py:1058] [5/0] Profiler function will be ignored 2024-08-20T23:40:58.0583629Z pass 2024-08-20T23:40:58.0837775Z TIMING: entire_frame_compile:6.60068 backend_compile:0.00123 total_wall_time:6.60068 2024-08-20T23:40:58.0838782Z STATS: call_* op count: 539 | FakeTensorMode.__torch_dispatch__:3512 | FakeTensor.__torch_dispatch__:1909 2024-08-20T23:40:58.0839691Z Dynamo produced 3 graphs covering 539 ops with 6 graph breaks (5 unique) 2024-08-20T23:41:01.8785012Z 2024-08-20T23:41:05.2066387Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:41:05.2066977Z loading model: 0it [00:03, ?it/s] 2024-08-20T23:41:05.2068575Z cuda train hf_GPT2 2024-08-20T23:41:14.5623878Z W0820 23:41:14.561000 36779 torch/_logging/_internal.py:1058] [5/0] Profiler function will be ignored 2024-08-20T23:41:20.5690592Z pass 2024-08-20T23:41:20.6517309Z TIMING: entire_frame_compile:10.50162 backend_compile:0.00184 total_wall_time:10.50162 2024-08-20T23:41:20.6518480Z STATS: call_* op count: 1087 | FakeTensorMode.__torch_dispatch__:6130 | FakeTensor.__torch_dispatch__:2680 2024-08-20T23:41:20.6519386Z Dynamo produced 3 graphs covering 1087 ops with 6 graph breaks (5 unique) 2024-08-20T23:41:24.5327750Z 2024-08-20T23:41:37.9422068Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:41:37.9422576Z loading model: 0it [00:13, ?it/s] 2024-08-20T23:41:37.9425777Z cuda train hf_GPT2_large 2024-08-20T23:41:37.9426310Z pass_due_to_skip 2024-08-20T23:41:38.4496240Z TIMING: 2024-08-20T23:41:38.4496676Z STATS: call_* op count: 0 2024-08-20T23:41:38.4497311Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:41:42.0244361Z 2024-08-20T23:41:43.6919206Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:41:43.6919799Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:41:43.6920244Z cuda train hf_Reformer 2024-08-20T23:41:54.8942513Z W0820 23:41:54.893000 36866 torch/_logging/_internal.py:1058] [32/0] Profiler function will be ignored 2024-08-20T23:41:58.5278814Z pass 2024-08-20T23:41:58.5923980Z TIMING: entire_frame_compile:6.84291 backend_compile:0.00178 total_wall_time:6.84291 2024-08-20T23:41:58.5926658Z STATS: call_* op count: 687 | FakeTensorMode.__torch_dispatch__:3404 | FakeTensor.__torch_dispatch__:1473 | attempt fast:21 | fast is_contiguous:21 2024-08-20T23:41:58.5928809Z Dynamo produced 17 graphs covering 687 ops with 25 graph breaks (9 unique) 2024-08-20T23:42:02.4890952Z 2024-08-20T23:42:06.8300872Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:06.8301403Z loading model: 0it [00:04, ?it/s] 2024-08-20T23:42:06.8301868Z cuda train hf_T5_base 2024-08-20T23:42:09.9596023Z WARNING:common:fp64 golden ref were not generated for hf_T5_base. Setting accuracy check to cosine 2024-08-20T23:42:12.0657367Z ERROR:common: 2024-08-20T23:42:12.0657895Z Traceback (most recent call last): 2024-08-20T23:42:12.0658899Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2753, in check_accuracy 2024-08-20T23:42:12.0659996Z correct_rerun_result = self.run_n_iterations( 2024-08-20T23:42:12.0661996Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2545, in run_n_iterations 2024-08-20T23:42:12.0664449Z self.model_iter_fn(mod, inputs, collect_outputs=False) 2024-08-20T23:42:12.0665779Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 445, in forward_and_backward_pass 2024-08-20T23:42:12.0667263Z self.grad_scaler.scale(loss).backward() 2024-08-20T23:42:12.0668723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_tensor.py", line 522, in backward 2024-08-20T23:42:12.0669789Z torch.autograd.backward( 2024-08-20T23:42:12.0671031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/autograd/__init__.py", line 347, in backward 2024-08-20T23:42:12.0672169Z _engine_run_backward( 2024-08-20T23:42:12.0673459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/autograd/graph.py", line 817, in _engine_run_backward 2024-08-20T23:42:12.0675097Z return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass 2024-08-20T23:42:12.0680113Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 192.00 MiB. GPU 0 has a total capacity of 21.98 GiB of which 88.44 MiB is free. Process 332030 has 21.88 GiB memory in use. Of the allocated memory 20.95 GiB is allocated by PyTorch, and 595.70 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 2024-08-20T23:42:12.1526707Z eager_2nd_run_OOM 2024-08-20T23:42:12.1558328Z TIMING: 2024-08-20T23:42:12.1558760Z STATS: call_* op count: 0 2024-08-20T23:42:12.1559573Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:42:15.7661759Z 2024-08-20T23:42:26.0881899Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:26.0882468Z loading model: 0it [00:10, ?it/s] 2024-08-20T23:42:26.0882992Z cuda train hf_T5_large 2024-08-20T23:42:26.0885523Z pass_due_to_skip 2024-08-20T23:42:26.5614954Z TIMING: 2024-08-20T23:42:26.5615354Z STATS: call_* op count: 0 2024-08-20T23:42:26.5615919Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T23:42:30.1637965Z 2024-08-20T23:42:31.3311234Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:31.3311852Z loading model: 0it [00:01, ?it/s] 2024-08-20T23:42:31.3312369Z cuda train hf_Whisper 2024-08-20T23:42:35.6316476Z W0820 23:42:35.630000 36997 torch/_logging/_internal.py:1058] [5/0] Profiler function will be ignored 2024-08-20T23:42:38.5496674Z pass 2024-08-20T23:42:38.5550923Z TIMING: entire_frame_compile:4.36481 backend_compile:0.00088 total_wall_time:4.36481 2024-08-20T23:42:38.5551906Z STATS: call_* op count: 346 | FakeTensorMode.__torch_dispatch__:2107 | FakeTensor.__torch_dispatch__:1318 2024-08-20T23:42:38.5552812Z Dynamo produced 3 graphs covering 346 ops with 6 graph breaks (5 unique) 2024-08-20T23:42:42.2995937Z 2024-08-20T23:42:42.5990824Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:42.5991388Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:42.5991840Z cuda train hf_distil_whisper 2024-08-20T23:42:42.6005394Z Traceback (most recent call last): 2024-08-20T23:42:42.6006203Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T23:42:42.6006932Z ) = runner.load_model( 2024-08-20T23:42:42.6007668Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/torchbench.py", line 302, in load_model 2024-08-20T23:42:42.6008456Z benchmark = benchmark_cls( 2024-08-20T23:42:42.6010737Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/util/model.py", line 24, in __call__ 2024-08-20T23:42:42.6011804Z obj = type.__call__(cls, *args, **kwargs) 2024-08-20T23:42:42.6012744Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_distil_whisper/__init__.py", line 12, in __init__ 2024-08-20T23:42:42.6013734Z raise NotImplementedError("Training is not implemented.") 2024-08-20T23:42:42.6014566Z NotImplementedError: Training is not implemented. 2024-08-20T23:42:42.6014928Z 2024-08-20T23:42:42.6015054Z model_fail_to_load 2024-08-20T23:42:46.0404023Z 2024-08-20T23:42:46.4928489Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:46.4928993Z loading model: 0it [00:00, ?it/s] 2024-08-20T23:42:46.4929444Z cuda train lennard_jones 2024-08-20T23:42:47.0187390Z W0820 23:42:47.017000 37075 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T23:42:47.4746409Z pass 2024-08-20T23:42:47.4747638Z TIMING: entire_frame_compile:0.73592 backend_compile:0.00026 total_wall_time:0.73592 2024-08-20T23:42:47.4748707Z STATS: call_* op count: 60 | FakeTensorMode.__torch_dispatch__:287 | FakeTensor.__torch_dispatch__:200 2024-08-20T23:42:47.4749692Z Dynamo produced 3 graphs covering 60 ops with 7 graph breaks (5 unique) 2024-08-20T23:42:48.2663848Z accuracy pass_rate=81.82% 2024-08-20T23:42:48.2667542Z calls_captured gmean=0.00x mean=447.394x 2024-08-20T23:42:48.2671034Z unique_graphs gmean=0.00x mean=3.303x 2024-08-20T23:42:48.2675344Z graph_breaks gmean=0.00x mean=6.545x 2024-08-20T23:42:48.2679292Z unique_graph_breaks gmean=0.00x mean=4.333x 2024-08-20T23:42:48.2683473Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T23:42:48.2687565Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T23:42:48.2691515Z cudagraph_skips gmean=0.00x mean=0.000x 2024-08-20T23:42:48.8589614Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamo_eager_torchbench_training.csv 2024-08-20T23:42:49.1942344Z torchrec_dlrm PASS 2024-08-20T23:42:49.1946460Z BERT_pytorch PASS 2024-08-20T23:42:49.1952130Z Background_Matting XFAIL 2024-08-20T23:42:49.1957168Z LearningToPaint PASS 2024-08-20T23:42:49.1962391Z Super_SloMo PASS 2024-08-20T23:42:49.1967710Z alexnet PASS 2024-08-20T23:42:49.1972957Z basic_gnn_edgecnn PASS 2024-08-20T23:42:49.1978093Z basic_gnn_gcn PASS 2024-08-20T23:42:49.1983530Z basic_gnn_gin PASS 2024-08-20T23:42:49.1988657Z basic_gnn_sage PASS 2024-08-20T23:42:49.1993498Z dcgan PASS 2024-08-20T23:42:49.1998264Z demucs PASS 2024-08-20T23:42:49.2002608Z densenet121 PASS 2024-08-20T23:42:49.2007248Z detectron2_maskrcnn_r_50_c4 XFAIL 2024-08-20T23:42:49.2011754Z dlrm PASS 2024-08-20T23:42:49.2016160Z drq PASS 2024-08-20T23:42:49.2020746Z fastNLP_Bert PASS 2024-08-20T23:42:49.2025182Z functorch_dp_cifar10 PASS 2024-08-20T23:42:49.2029657Z functorch_maml_omniglot PASS 2024-08-20T23:42:49.2033896Z hf_Albert PASS 2024-08-20T23:42:49.2039008Z hf_Bart PASS 2024-08-20T23:42:49.2043596Z hf_Bert PASS 2024-08-20T23:42:49.2048066Z hf_Bert_large PASS 2024-08-20T23:42:49.2052312Z hf_BigBird PASS 2024-08-20T23:42:49.2056620Z hf_DistilBert PASS 2024-08-20T23:42:49.2061033Z hf_GPT2 PASS 2024-08-20T23:42:49.2066163Z hf_GPT2_large XFAIL 2024-08-20T23:42:49.2070207Z hf_Reformer PASS 2024-08-20T23:42:49.2074648Z hf_T5_base XFAIL 2024-08-20T23:42:49.2079068Z hf_T5_large XFAIL 2024-08-20T23:42:49.2083311Z hf_Whisper PASS 2024-08-20T23:42:49.2087768Z hf_distil_whisper XFAIL 2024-08-20T23:42:49.2092152Z lennard_jones PASS 2024-08-20T23:42:49.2617509Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamo_eager_torchbench_training.csv 2024-08-20T23:42:49.5961177Z torchrec_dlrm PASS 2024-08-20T23:42:49.5964815Z BERT_pytorch PASS 2024-08-20T23:42:49.5969655Z Background_Matting PASS 2024-08-20T23:42:49.5974727Z LearningToPaint PASS 2024-08-20T23:42:49.5979554Z Super_SloMo PASS 2024-08-20T23:42:49.5984446Z alexnet PASS 2024-08-20T23:42:49.5989321Z basic_gnn_edgecnn PASS 2024-08-20T23:42:49.5993885Z basic_gnn_gcn PASS 2024-08-20T23:42:49.5998641Z basic_gnn_gin PASS 2024-08-20T23:42:49.6003160Z basic_gnn_sage PASS 2024-08-20T23:42:49.6007793Z dcgan PASS 2024-08-20T23:42:49.6012625Z demucs PASS 2024-08-20T23:42:49.6017078Z densenet121 PASS 2024-08-20T23:42:49.6021723Z detectron2_maskrcnn_r_50_c4 PASS 2024-08-20T23:42:49.6026448Z dlrm PASS 2024-08-20T23:42:49.6031201Z drq PASS 2024-08-20T23:42:49.6036016Z fastNLP_Bert PASS 2024-08-20T23:42:49.6041238Z functorch_dp_cifar10 PASS 2024-08-20T23:42:49.6045686Z functorch_maml_omniglot PASS 2024-08-20T23:42:49.6050518Z hf_Albert PASS 2024-08-20T23:42:49.6054944Z hf_Bart PASS 2024-08-20T23:42:49.6059606Z hf_Bert PASS 2024-08-20T23:42:49.6064253Z hf_Bert_large PASS 2024-08-20T23:42:49.6069080Z hf_BigBird PASS 2024-08-20T23:42:49.6073757Z hf_DistilBert PASS 2024-08-20T23:42:49.6078356Z hf_GPT2 PASS 2024-08-20T23:42:49.6082825Z hf_GPT2_large PASS 2024-08-20T23:42:49.6087372Z hf_Reformer PASS 2024-08-20T23:42:49.6091942Z hf_T5_base PASS 2024-08-20T23:42:49.6096593Z hf_T5_large PASS 2024-08-20T23:42:49.6101158Z hf_Whisper PASS 2024-08-20T23:42:49.6105997Z hf_distil_whisper PASS 2024-08-20T23:42:49.6110492Z lennard_jones PASS 2024-08-20T23:42:49.6648244Z + cleanup_workspace 2024-08-20T23:42:49.6649446Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T23:42:49.6650559Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T23:42:49.6651489Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T23:42:49.6652140Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T23:42:49.6652971Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T23:42:49.6653855Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T23:42:49.6654547Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T23:42:50.7302298Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T23:42:50.7302902Z cat test/**/*_toprint.log || true 2024-08-20T23:42:50.7317202Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:42:50.7317716Z env: 2024-08-20T23:42:50.7318002Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:50.7318468Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:50.7319228Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:50.7319897Z ##[endgroup] 2024-08-20T23:42:50.7417077Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T23:42:50.7449860Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:42:50.7450364Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:42:50.7458807Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:42:50.7459318Z env: 2024-08-20T23:42:50.7459785Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:50.7460253Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:50.7461014Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:50.7461693Z MONITOR_SCRIPT_PID: 291821 2024-08-20T23:42:50.7462055Z ##[endgroup] 2024-08-20T23:42:50.7655902Z Prepare all required actions 2024-08-20T23:42:50.7656362Z Getting action download info 2024-08-20T23:42:50.9661151Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T23:42:51.1262844Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T23:42:51.1263320Z with: 2024-08-20T23:42:51.1263877Z file-suffix: test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876 2024-08-20T23:42:51.1264571Z s3-bucket: gha-artifacts 2024-08-20T23:42:51.1264916Z env: 2024-08-20T23:42:51.1265195Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:51.1265661Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:51.1266437Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:51.1267100Z ##[endgroup] 2024-08-20T23:42:51.1294492Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T23:42:51.1295137Z # Remove any previous test jsons if they exist 2024-08-20T23:42:51.1295641Z rm -f test-jsons-*.zip 2024-08-20T23:42:51.1296169Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T23:42:51.1306038Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:42:51.1306552Z env: 2024-08-20T23:42:51.1306830Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:51.1307297Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:51.1308045Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:51.1308981Z FILE_SUFFIX: test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876 2024-08-20T23:42:51.1309644Z ##[endgroup] 2024-08-20T23:42:51.1593701Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T23:42:51.1623806Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T23:42:51.1624848Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T23:42:51.1632232Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T23:42:51.1638926Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T23:42:51.1672174Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T23:42:51.1672826Z # Remove any previous test reports if they exist 2024-08-20T23:42:51.1673368Z rm -f test-reports-*.zip 2024-08-20T23:42:51.1673973Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T23:42:51.1683031Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:42:51.1683538Z env: 2024-08-20T23:42:51.1683840Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:51.1684319Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:51.1685079Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:51.1686017Z FILE_SUFFIX: test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876 2024-08-20T23:42:51.1686679Z ##[endgroup] 2024-08-20T23:42:51.1951969Z adding: test/test-reports/inference_torchbench.csv (deflated 73%) 2024-08-20T23:42:51.1956514Z adding: test/test-reports/inference_torchbench_graph_breaks.csv (deflated 92%) 2024-08-20T23:42:51.1958916Z adding: test/test-reports/inference_torchbench_graph_break_deduped.csv (deflated 80%) 2024-08-20T23:42:51.1959949Z adding: test/test-reports/training_torchbench.csv (deflated 64%) 2024-08-20T23:42:51.1966375Z adding: test/test-reports/training_torchbench_graph_breaks.csv (deflated 96%) 2024-08-20T23:42:51.1967707Z adding: test/test-reports/training_torchbench_graph_break_deduped.csv (deflated 76%) 2024-08-20T23:42:51.2002341Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T23:42:51.2002981Z # Remove any previous usage logs if they exist 2024-08-20T23:42:51.2003485Z rm -f logs-*.zip 2024-08-20T23:42:51.2004377Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T23:42:51.2005179Z # so check to see if the file exists first 2024-08-20T23:42:51.2005699Z if [ -f 'usage_log.txt' ]; then 2024-08-20T23:42:51.2006225Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T23:42:51.2006726Z fi 2024-08-20T23:42:51.2007102Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T23:42:51.2007687Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T23:42:51.2008285Z fi 2024-08-20T23:42:51.2017392Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:42:51.2017897Z env: 2024-08-20T23:42:51.2018186Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:51.2018654Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:51.2019458Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:51.2020390Z FILE_SUFFIX: test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876 2024-08-20T23:42:51.2021069Z ##[endgroup] 2024-08-20T23:42:51.2115933Z adding: usage_log.txt (deflated 92%) 2024-08-20T23:42:51.2207469Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T23:42:51.2208195Z # Remove any previous debugging artifacts if they exist 2024-08-20T23:42:51.2208754Z rm -f debug-*.zip 2024-08-20T23:42:51.2209147Z if [ -d 'test/debug' ]; then 2024-08-20T23:42:51.2209660Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T23:42:51.2210150Z fi 2024-08-20T23:42:51.2218396Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:42:51.2218909Z env: 2024-08-20T23:42:51.2219201Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:51.2219666Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:51.2220426Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:51.2221375Z FILE_SUFFIX: test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876 2024-08-20T23:42:51.2222041Z ##[endgroup] 2024-08-20T23:42:51.2320373Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:42:51.2321021Z with: 2024-08-20T23:42:51.2321443Z s3-bucket: gha-artifacts 2024-08-20T23:42:51.2322136Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:42:51.2322889Z retention-days: 14 2024-08-20T23:42:51.2323412Z if-no-files-found: warn 2024-08-20T23:42:51.2323966Z path: test-jsons-*.zip 2024-08-20T23:42:51.2324452Z name: artifact 2024-08-20T23:42:51.2324882Z region: us-east-1 2024-08-20T23:42:51.2325318Z env: 2024-08-20T23:42:51.2325711Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:51.2326412Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:51.2327533Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:51.2328562Z ##[endgroup] 2024-08-20T23:42:51.5863391Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:42:51.5864144Z With the provided path, there will be 1 file uploaded 2024-08-20T23:42:51.5864945Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:42:51.5935787Z Starting upload of test-jsons-test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876.zip 2024-08-20T23:42:51.7103508Z Finished upload of test-jsons-test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876.zip 2024-08-20T23:42:51.7382866Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:42:51.7383324Z with: 2024-08-20T23:42:51.7383614Z s3-bucket: gha-artifacts 2024-08-20T23:42:51.7384062Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:42:51.7384734Z retention-days: 14 2024-08-20T23:42:51.7385077Z if-no-files-found: error 2024-08-20T23:42:51.7385437Z path: test-reports-*.zip 2024-08-20T23:42:51.7385795Z name: artifact 2024-08-20T23:42:51.7386104Z region: us-east-1 2024-08-20T23:42:51.7386408Z env: 2024-08-20T23:42:51.7386681Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:51.7387315Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:51.7388081Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:51.7388791Z ##[endgroup] 2024-08-20T23:42:52.0956426Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:42:52.0957127Z With the provided path, there will be 1 file uploaded 2024-08-20T23:42:52.0957751Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:42:52.1014873Z Starting upload of test-reports-test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876.zip 2024-08-20T23:42:52.1947787Z Finished upload of test-reports-test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876.zip 2024-08-20T23:42:52.2233311Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:42:52.2233764Z with: 2024-08-20T23:42:52.2234068Z s3-bucket: gha-artifacts 2024-08-20T23:42:52.2234517Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:42:52.2235028Z retention-days: 14 2024-08-20T23:42:52.2235371Z if-no-files-found: ignore 2024-08-20T23:42:52.2235743Z path: logs-*.zip 2024-08-20T23:42:52.2236056Z name: artifact 2024-08-20T23:42:52.2236363Z region: us-east-1 2024-08-20T23:42:52.2237054Z env: 2024-08-20T23:42:52.2237351Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:52.2237819Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:52.2238577Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:52.2239239Z ##[endgroup] 2024-08-20T23:42:52.5780470Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:42:52.5781080Z With the provided path, there will be 1 file uploaded 2024-08-20T23:42:52.5781711Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:42:52.5838445Z Starting upload of logs-test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876.zip 2024-08-20T23:42:52.7102355Z Finished upload of logs-test-dynamo_eager_torchbench-1-2-linux.g5.4xlarge.nvidia.gpu_29026297876.zip 2024-08-20T23:42:52.7387137Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:42:52.7387584Z with: 2024-08-20T23:42:52.7387878Z s3-bucket: gha-artifacts 2024-08-20T23:42:52.7388324Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T23:42:52.7388808Z retention-days: 14 2024-08-20T23:42:52.7389154Z if-no-files-found: ignore 2024-08-20T23:42:52.7389521Z path: debug-*.zip 2024-08-20T23:42:52.7389846Z name: artifact 2024-08-20T23:42:52.7390154Z region: us-east-1 2024-08-20T23:42:52.7390464Z env: 2024-08-20T23:42:52.7390749Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:52.7391210Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:52.7391972Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:52.7392631Z ##[endgroup] 2024-08-20T23:42:53.0859971Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T23:42:53.1140388Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T23:42:53.1140887Z # shellcheck disable=SC2156 2024-08-20T23:42:53.1141693Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T23:42:53.1151102Z shell: /usr/bin/bash -e {0} 2024-08-20T23:42:53.1151467Z env: 2024-08-20T23:42:53.1151755Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:53.1152217Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:53.1152967Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:53.1153629Z ##[endgroup] 2024-08-20T23:42:53.3845848Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T23:42:53.3846434Z with: 2024-08-20T23:42:53.3846714Z env: 2024-08-20T23:42:53.3847011Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:53.3847478Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:53.3848237Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:53.3848906Z ##[endgroup] 2024-08-20T23:42:53.3868957Z ##[group]Run set -eou pipefail 2024-08-20T23:42:53.3869433Z set -eou pipefail 2024-08-20T23:42:53.3869806Z  2024-08-20T23:42:53.3870333Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T23:42:53.3870974Z for _ in $(seq 1440); do 2024-08-20T23:42:53.3871463Z  # Break if no ssh session exists anymore 2024-08-20T23:42:53.3871968Z  if [ "$(who)" = "" ]; then 2024-08-20T23:42:53.3872391Z  break 2024-08-20T23:42:53.3872744Z  fi 2024-08-20T23:42:53.3873047Z  echo "." 2024-08-20T23:42:53.3873382Z  sleep 5 2024-08-20T23:42:53.3873711Z done 2024-08-20T23:42:53.3882600Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:42:53.3883106Z env: 2024-08-20T23:42:53.3883399Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:53.3883869Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:53.3884630Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:53.3885297Z ##[endgroup] 2024-08-20T23:42:53.3916174Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T23:42:53.3988467Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:42:53.3989245Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:42:53.3989878Z # shellcheck disable=SC2046 2024-08-20T23:42:53.3990392Z docker stop $(docker ps -q) || true 2024-08-20T23:42:53.3990895Z # Prune all of the docker images 2024-08-20T23:42:53.3991352Z docker system prune -af 2024-08-20T23:42:53.4000146Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:42:53.4000654Z env: 2024-08-20T23:42:53.4000940Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:42:53.4001405Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T23:42:53.4002165Z DOCKER_CONTAINER_ID: b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:53.4002830Z ##[endgroup] 2024-08-20T23:42:54.1453590Z b5dc86f43b5d 2024-08-20T23:42:58.1830839Z Deleted Containers: 2024-08-20T23:42:58.1831434Z b5dc86f43b5df9a26f74f244bcfc7a13d8eefa759bff376873be7ab155e40192 2024-08-20T23:42:58.1831882Z 2024-08-20T23:43:06.3626201Z Deleted Images: 2024-08-20T23:43:06.3628037Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T23:43:06.3630280Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T23:43:06.3632113Z deleted: sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb 2024-08-20T23:43:06.3633105Z deleted: sha256:c2ce10014dd98f6abbfbf2265f9780a8204c2a7a27463e61ee565573a23d141d 2024-08-20T23:43:06.3633970Z deleted: sha256:ae249c0e6e6013bd6ffea2ca0b0160c015c9ebf7c03070bc2f983a9ff2415bf5 2024-08-20T23:43:06.3634798Z deleted: sha256:8a73883c1067c0e29ff128f553518682b47ed9707d2f35e7d5138e2caf029d61 2024-08-20T23:43:06.3635614Z deleted: sha256:9c040db729fa084584aaf5573082408a2196bc64e5952270c79dc610ed4680b6 2024-08-20T23:43:06.3636679Z deleted: sha256:17311eeeb8b71b16699880c0f646e185107496c4d41da5d23370603768ec030c 2024-08-20T23:43:06.3637769Z deleted: sha256:07cc10bece8501b1c8035d4385ad9e9867dd6e92fa32218a4dcb3e42af7e0411 2024-08-20T23:43:06.3639321Z deleted: sha256:f3e32d05099d1a3115a4722bc2e94a48752c3e965446773673808252671bb967 2024-08-20T23:43:06.3640148Z deleted: sha256:be7c3f41475e3e72ffe3ac2f92ab2f4cf49d5496b2aae738942ac3bb687d742c 2024-08-20T23:43:06.3640994Z deleted: sha256:4fcac490be825129b468cf458089f1fcf8fd56afc44ef4ab27c2eff41ab1ac90 2024-08-20T23:43:06.3641830Z deleted: sha256:31c66d80a2a4792897f267d363c0cf84fb821c5d4a275ab1636f778fd0284c08 2024-08-20T23:43:06.3642643Z deleted: sha256:8a8bcb6085e7e438e5533e8a5613e2ae5888543853568d023af3e2623a1593a9 2024-08-20T23:43:06.3643461Z deleted: sha256:2f6cb60029c79d11567b738f5bdbfb9cb1838cc0f194c4a5e6ae685178d2ed34 2024-08-20T23:43:06.3644285Z deleted: sha256:e546391804d3313f558e1ac16e93cb29a417a92e4eccaf0d4bd12af3bbcf39e3 2024-08-20T23:43:06.3645105Z deleted: sha256:1623b2aa16f2bdc6b572f6799498d9a92fa8114d07c440599b17aea72e22f8b1 2024-08-20T23:43:06.3645937Z deleted: sha256:c0386ca58b7cb7afee015b7e58d2f70525eaf73d6da2cc63868b20e14b1f9497 2024-08-20T23:43:06.3646767Z deleted: sha256:dcbf534ff289186e246531925fed90f72516eb3d63663bad234d611cdc025d62 2024-08-20T23:43:06.3647597Z deleted: sha256:0bab537af579f56c9cd51c7073fd53635fffb424b363236e0de38a7dd0e60ae8 2024-08-20T23:43:06.3648423Z deleted: sha256:d85a852ae1ee027e2fc17240ed68b87957b0a6d210c9119752d8979a0dcd55cf 2024-08-20T23:43:06.3649338Z deleted: sha256:f3821d761b555362e44cf9a1447d8fd2d95b37d54aea67c7519cd9e5db0c4d7e 2024-08-20T23:43:06.3650165Z deleted: sha256:c05e84a545db5dfcb0847c73d2b0f4749fa67aabce05705632ff7f76c84d4688 2024-08-20T23:43:06.3650997Z deleted: sha256:27054eae7ca67b2bd5ca0fd3f4488bf5c5f78e95487d8c45616f3d753d48a8b5 2024-08-20T23:43:06.3651818Z deleted: sha256:9804139f6ae97796aaa993fe4b89ea0cf111eb4d30a79f4c87c339c2e1e04fa0 2024-08-20T23:43:06.3652640Z deleted: sha256:ad6ae2fc594aaa359ff4e287323836570454eb24dd162930655e414d1d77c6dd 2024-08-20T23:43:06.3653700Z deleted: sha256:9cfbb36e328af52a248853a46aab88e266c09e2b44a03e0a42194a031b9b9453 2024-08-20T23:43:06.3654580Z deleted: sha256:a8a842bd7f37b8d50b6d12d6f95f46bbba57953db466fcbde9ba522625f024ef 2024-08-20T23:43:06.3655424Z deleted: sha256:da9738654b3c86f89ebeed2560f1bae093f1219f1eff90b0394b7f211ecabd04 2024-08-20T23:43:06.3656261Z deleted: sha256:cc683bf0cb245394e0f568984b2dd4fce1b6a3a7fe7c44734bae1aa1caaa3251 2024-08-20T23:43:06.3657091Z deleted: sha256:965f220e71c7d1ec19aaf41b1529ab287beb9d86eb8482c4c14094485ab34475 2024-08-20T23:43:06.3657911Z deleted: sha256:da32ecec41a73ce7dd29464f90c7951510a18cd378753659a78cd7fabd10dc36 2024-08-20T23:43:06.3658736Z deleted: sha256:d828d187cee9ad48dbc3702fb61d359619dd8a1ed582878680b51686bf117e5a 2024-08-20T23:43:06.3659551Z deleted: sha256:712111f4b12ff8e2787597d5fdcb043bd0824e55ddd97f262ef550506ca195f0 2024-08-20T23:43:06.3660364Z deleted: sha256:a064eea405958ad14dc6c794e5df9fd1d4515424272801ec559d45a59317309a 2024-08-20T23:43:06.3661177Z deleted: sha256:e124382d136bd5e9a8b409ca79c1f1ef76458deaf444b1b2060d3fa99dbeb506 2024-08-20T23:43:06.3661998Z deleted: sha256:6e10fe757cc190a1b31b47010f4f7610e02cb1b4b438f190dcd301aa4fe2278e 2024-08-20T23:43:06.3662940Z deleted: sha256:4cea352e10f46ca4729f96ac6c4600fc53d4c36e7658c2e0f19f4f309cd4a2a7 2024-08-20T23:43:06.3663761Z deleted: sha256:518bba00fe1d0933053066d21cac0235c9fb485abbc41557ec184b293095c924 2024-08-20T23:43:06.3664630Z deleted: sha256:a64f9ca42e9f27ff2690c15a5c84312e6efaccad45668d9b496cb542165cfb5a 2024-08-20T23:43:06.3665454Z deleted: sha256:096ea0fa14a6c420658f5276b8a8f5d264c0de7e18161ceda2cfc2f4a92cd9bf 2024-08-20T23:43:06.3666277Z deleted: sha256:6f91c94ce6e3f8034c91c66ee25c66a3d85dba9d813b749bd6d2000942a99edc 2024-08-20T23:43:06.3667106Z deleted: sha256:8f8d255eeae546cead8f9a94d237fa57a99940c0f5d51bc29a9d41c7df9cffd9 2024-08-20T23:43:06.3667943Z deleted: sha256:fe05276f4a49f91eef56bfbc3da87cfff5eb116a827fa2d30efe42ce1c134b40 2024-08-20T23:43:06.3668756Z deleted: sha256:5f83cd75d0669c813d4f58978c92e14305cd8f6a07eb9a60665a583460fa7b59 2024-08-20T23:43:06.3669559Z deleted: sha256:8f7ab6366169d39d8953129f650f1927d2fb1328b3f11e2fd9076adb5f46b93a 2024-08-20T23:43:06.3670511Z deleted: sha256:4146cfd5dba0f5308954749f98596b14a1d8f6d23bd93f7ad86c6c3d72239328 2024-08-20T23:43:06.3672417Z deleted: sha256:c6d83cb9ccc606ff63bb3c4d5a5e1ff1450c619b5aae494303a94ae25971cca0 2024-08-20T23:43:06.3673259Z deleted: sha256:71e037e0a3e0963c4565534a7387c082afcb04475d3ac629acf7a0ab2f49fadc 2024-08-20T23:43:06.3674074Z deleted: sha256:4b31135c564686bdaf23262b901237880aea635655721cc7e8c95dc42b05e06d 2024-08-20T23:43:06.3674972Z deleted: sha256:d09bd5c2f35cedcb846543941ee64ce1f684b52e4285696c93b3fb2032da8579 2024-08-20T23:43:06.3675819Z deleted: sha256:4de7fb7d4fc22212aae5fc1ceb637526916d0cced240d7c93d5edd6b0ea28013 2024-08-20T23:43:06.3676660Z deleted: sha256:69890e3e82746e10aff2f1084f11afb4b95a4a6c26b30f64b7001bc0a5fe32bf 2024-08-20T23:43:06.3677472Z deleted: sha256:0156df53b3224e645336ea52b4bb816f1e8d8c9b591748cd645e5e5649fa507b 2024-08-20T23:43:06.3678304Z deleted: sha256:0664624cb9cc4a386f31bf49919a8d8a11d9fea428d05585d9e6228f754bc7cd 2024-08-20T23:43:06.3679154Z deleted: sha256:8c2ddd5fa535c768c058a23f5cc4b6c64cdd71bb7458491282aabc7f7bb94d69 2024-08-20T23:43:06.3680014Z deleted: sha256:8dd03dbe0e1c5f47dd333074dff4bd00415f2f63727a65bf17866774e2ebbd0d 2024-08-20T23:43:06.3680849Z deleted: sha256:3c56b63e094bd18fad9da7a541d537210bca3b730c590d3641213259f6908d6f 2024-08-20T23:43:06.3681682Z deleted: sha256:51484fef07bd1266d4d6ff3d767751172660fc4cc6a1ffe883523a63c985e31e 2024-08-20T23:43:06.3682508Z deleted: sha256:03e83c94023982ab7dd8dc65a6df2024e342f1b760c91ec7a680ee45824869c7 2024-08-20T23:43:06.3683330Z deleted: sha256:58440ab3e8175b5de94a76485c8689a6dc186015475ebf4b386ebba309c5da34 2024-08-20T23:43:06.3684206Z deleted: sha256:9a980f3383e8aa511f0ef68982cb6dfc53b38d1ac8c3d673cb3ba3d45158c91e 2024-08-20T23:43:06.3685045Z deleted: sha256:cb0269c9cf1c47b6c4188ca7be4db261b5d3fc4b4a5153629e222dba7b1f9f98 2024-08-20T23:43:06.3685900Z deleted: sha256:41f1ba870cdffc05e0bf8d87b640fb9d4c153d7f4f3b775f37f94e9f3d584aad 2024-08-20T23:43:06.3686743Z deleted: sha256:7b502ab352f48e8b6eb5e6824a29f308952a5ee7c7d987562e1df7e47639d7f1 2024-08-20T23:43:06.3687573Z deleted: sha256:6ba5217e18738660e6ca8b7a7ec88845e70b0fd781301520bf46c71d99df3144 2024-08-20T23:43:06.3688410Z deleted: sha256:dd5a1c233594780163726bdbbf5de00f7b6b8dca1dace8a001340714c54baedb 2024-08-20T23:43:06.3689252Z deleted: sha256:c27f0db27da73e01f67f6cb78d10f355e350d829479ba4351d82055bb29f6597 2024-08-20T23:43:06.3690088Z deleted: sha256:4e3ce3c854f35c08be3dd95e0d6e718512a885e4614291d7d8b7cd23f48b7dd8 2024-08-20T23:43:06.3690924Z deleted: sha256:c2d6fb088bbec16e60cf0ad74b016f7287c92e659ec2db01271bc309b4dc6d83 2024-08-20T23:43:06.3691761Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-08-20T23:43:06.3692596Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-08-20T23:43:06.3693418Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-08-20T23:43:06.3694292Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-08-20T23:43:06.3695152Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-08-20T23:43:06.3695989Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-08-20T23:43:06.3696830Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-08-20T23:43:06.3697683Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-08-20T23:43:06.3698523Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-08-20T23:43:06.3699373Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-08-20T23:43:06.3700204Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-20T23:43:06.3700700Z 2024-08-20T23:43:06.3700859Z Total reclaimed space: 49.69GB 2024-08-20T23:43:06.3769951Z Post job cleanup. 2024-08-20T23:43:06.3820747Z Post job cleanup. 2024-08-20T23:43:06.4716572Z [command]/usr/bin/git version 2024-08-20T23:43:06.4771780Z git version 2.40.1 2024-08-20T23:43:06.4802895Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/0ca4d004-8eae-4195-9784-c8d22dd643e9/.gitconfig' 2024-08-20T23:43:06.4813805Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/0ca4d004-8eae-4195-9784-c8d22dd643e9' before making global git config changes 2024-08-20T23:43:06.4815033Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T23:43:06.4818592Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T23:43:06.4872076Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T23:43:06.4916951Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T23:43:06.5294542Z Entering 'android/libs/fbjni' 2024-08-20T23:43:06.5364702Z Entering 'third_party/FP16' 2024-08-20T23:43:06.5434610Z Entering 'third_party/FXdiv' 2024-08-20T23:43:06.5503382Z Entering 'third_party/NNPACK' 2024-08-20T23:43:06.5573560Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:43:06.5643119Z Entering 'third_party/XNNPACK' 2024-08-20T23:43:06.5728218Z Entering 'third_party/benchmark' 2024-08-20T23:43:06.5798160Z Entering 'third_party/cpp-httplib' 2024-08-20T23:43:06.5867221Z Entering 'third_party/cpuinfo' 2024-08-20T23:43:06.5936319Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:43:06.6005587Z Entering 'third_party/cutlass' 2024-08-20T23:43:06.6082488Z Entering 'third_party/eigen' 2024-08-20T23:43:06.6155188Z Entering 'third_party/fbgemm' 2024-08-20T23:43:06.6223242Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:43:06.6290374Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:43:06.6358234Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:43:06.6432808Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:43:06.6504756Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:43:06.6578754Z Entering 'third_party/flatbuffers' 2024-08-20T23:43:06.6651862Z Entering 'third_party/fmt' 2024-08-20T23:43:06.6720708Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:43:06.6791369Z Entering 'third_party/gloo' 2024-08-20T23:43:06.6859688Z Entering 'third_party/googletest' 2024-08-20T23:43:06.6928889Z Entering 'third_party/ideep' 2024-08-20T23:43:06.6996711Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:43:06.7072105Z Entering 'third_party/ittapi' 2024-08-20T23:43:06.7139664Z Entering 'third_party/kineto' 2024-08-20T23:43:06.7206468Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:43:06.7272250Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:43:06.7342355Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:43:06.7410725Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:43:06.7479303Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:43:06.7545653Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:43:06.7616411Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:43:06.7685153Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:43:06.7757897Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:43:06.7827616Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:43:06.7897919Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:43:06.7965494Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:43:06.8035803Z Entering 'third_party/mimalloc' 2024-08-20T23:43:06.8103450Z Entering 'third_party/nccl/nccl' 2024-08-20T23:43:06.8172923Z Entering 'third_party/nlohmann' 2024-08-20T23:43:06.8243045Z Entering 'third_party/onnx' 2024-08-20T23:43:06.8326019Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:43:06.8397065Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:43:06.8470772Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:43:06.8540981Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:43:06.8607190Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:43:06.8675744Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:43:06.8744946Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:43:06.8814301Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:43:06.8879166Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:43:06.8951823Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:43:06.9018582Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:43:06.9088818Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:43:06.9157237Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:43:06.9251000Z Entering 'third_party/pocketfft' 2024-08-20T23:43:06.9324646Z Entering 'third_party/protobuf' 2024-08-20T23:43:06.9393036Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:43:06.9457835Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:43:06.9532301Z Entering 'third_party/psimd' 2024-08-20T23:43:06.9600360Z Entering 'third_party/pthreadpool' 2024-08-20T23:43:06.9666788Z Entering 'third_party/pybind11' 2024-08-20T23:43:06.9732305Z Entering 'third_party/python-peachpy' 2024-08-20T23:43:06.9798086Z Entering 'third_party/sleef' 2024-08-20T23:43:06.9864134Z Entering 'third_party/tensorpipe' 2024-08-20T23:43:06.9928251Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:43:06.9994843Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:43:07.0060296Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:43:07.0124644Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:43:07.0190280Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:43:07.0283555Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T23:43:07.0319599Z http.https://github.com/.extraheader 2024-08-20T23:43:07.0327969Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T23:43:07.0375981Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T23:43:07.0747588Z Entering 'android/libs/fbjni' 2024-08-20T23:43:07.0791489Z http.https://github.com/.extraheader 2024-08-20T23:43:07.0836631Z Entering 'third_party/FP16' 2024-08-20T23:43:07.0881939Z http.https://github.com/.extraheader 2024-08-20T23:43:07.0926286Z Entering 'third_party/FXdiv' 2024-08-20T23:43:07.0972540Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1017399Z Entering 'third_party/NNPACK' 2024-08-20T23:43:07.1062568Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1107895Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:43:07.1153070Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1197636Z Entering 'third_party/XNNPACK' 2024-08-20T23:43:07.1242367Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1302220Z Entering 'third_party/benchmark' 2024-08-20T23:43:07.1347263Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1390464Z Entering 'third_party/cpp-httplib' 2024-08-20T23:43:07.1435442Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1479495Z Entering 'third_party/cpuinfo' 2024-08-20T23:43:07.1523465Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1568547Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:43:07.1611772Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1656004Z Entering 'third_party/cutlass' 2024-08-20T23:43:07.1699398Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1751554Z Entering 'third_party/eigen' 2024-08-20T23:43:07.1794841Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1840994Z Entering 'third_party/fbgemm' 2024-08-20T23:43:07.1884295Z http.https://github.com/.extraheader 2024-08-20T23:43:07.1928159Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:43:07.1971135Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2014940Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:43:07.2062406Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2105957Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:43:07.2155882Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2207130Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:43:07.2249979Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2292910Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:43:07.2340833Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2386523Z Entering 'third_party/flatbuffers' 2024-08-20T23:43:07.2430708Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2478188Z Entering 'third_party/fmt' 2024-08-20T23:43:07.2522811Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2567759Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:43:07.2611854Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2656233Z Entering 'third_party/gloo' 2024-08-20T23:43:07.2700040Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2744312Z Entering 'third_party/googletest' 2024-08-20T23:43:07.2787329Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2830896Z Entering 'third_party/ideep' 2024-08-20T23:43:07.2877779Z http.https://github.com/.extraheader 2024-08-20T23:43:07.2919214Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:43:07.2962316Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3019530Z Entering 'third_party/ittapi' 2024-08-20T23:43:07.3064880Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3108485Z Entering 'third_party/kineto' 2024-08-20T23:43:07.3156356Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3200009Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:43:07.3242268Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3285306Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:43:07.3328238Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3374859Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:43:07.3417444Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3462226Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:43:07.3504742Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3549088Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:43:07.3591749Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3634092Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:43:07.3677276Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3724797Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:43:07.3769038Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3814312Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:43:07.3857964Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3901994Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:43:07.3944866Z http.https://github.com/.extraheader 2024-08-20T23:43:07.3989980Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:43:07.4032480Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4080366Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:43:07.4122724Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4166426Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:43:07.4212727Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4264081Z Entering 'third_party/mimalloc' 2024-08-20T23:43:07.4308304Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4352662Z Entering 'third_party/nccl/nccl' 2024-08-20T23:43:07.4400632Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4444949Z Entering 'third_party/nlohmann' 2024-08-20T23:43:07.4488661Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4533139Z Entering 'third_party/onnx' 2024-08-20T23:43:07.4581503Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4639780Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:43:07.4682834Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4727714Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:43:07.4769883Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4819986Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:43:07.4865720Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4910902Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:43:07.4954274Z http.https://github.com/.extraheader 2024-08-20T23:43:07.4997099Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:43:07.5040457Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5084800Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:43:07.5127637Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5171620Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:43:07.5214082Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5262842Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:43:07.5305026Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5349666Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:43:07.5391875Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5434205Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:43:07.5477126Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5518427Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:43:07.5561981Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5608353Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:43:07.5651534Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5699310Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:43:07.5747026Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5813327Z Entering 'third_party/pocketfft' 2024-08-20T23:43:07.5858070Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5902026Z Entering 'third_party/protobuf' 2024-08-20T23:43:07.5946694Z http.https://github.com/.extraheader 2024-08-20T23:43:07.5992846Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:43:07.6034893Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6078906Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:43:07.6121597Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6169485Z Entering 'third_party/psimd' 2024-08-20T23:43:07.6212957Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6257205Z Entering 'third_party/pthreadpool' 2024-08-20T23:43:07.6301040Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6346519Z Entering 'third_party/pybind11' 2024-08-20T23:43:07.6389997Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6434110Z Entering 'third_party/python-peachpy' 2024-08-20T23:43:07.6478165Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6521737Z Entering 'third_party/sleef' 2024-08-20T23:43:07.6567245Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6610385Z Entering 'third_party/tensorpipe' 2024-08-20T23:43:07.6654898Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6698068Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:43:07.6740926Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6783382Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:43:07.6830699Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6873325Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:43:07.6915826Z http.https://github.com/.extraheader 2024-08-20T23:43:07.6959283Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:43:07.7002075Z http.https://github.com/.extraheader 2024-08-20T23:43:07.7045835Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:43:07.7088393Z http.https://github.com/.extraheader 2024-08-20T23:43:07.7252367Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T23:43:07.7274125Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T23:43:07.7282209Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:43:07.7282741Z ##[endgroup] 2024-08-20T23:43:16.1573545Z Cleaning up orphan processes