2024-11-01T16:44:39.2073449Z Current runner version: '2.320.0' 2024-11-01T16:44:39.2079979Z Runner name: 'i-0fb054b0c783c31fa' 2024-11-01T16:44:39.2080709Z Runner group name: 'Default' 2024-11-01T16:44:39.2081528Z Machine name: 'ip-10-0-54-33' 2024-11-01T16:44:39.2098005Z Testing runner upgrade compatibility 2024-11-01T16:44:39.3130044Z ##[group]GITHUB_TOKEN Permissions 2024-11-01T16:44:39.3132536Z Actions: read 2024-11-01T16:44:39.3133144Z Attestations: read 2024-11-01T16:44:39.3133631Z Checks: read 2024-11-01T16:44:39.3134096Z Contents: read 2024-11-01T16:44:39.3134560Z Deployments: read 2024-11-01T16:44:39.3135052Z Discussions: read 2024-11-01T16:44:39.3135534Z Issues: read 2024-11-01T16:44:39.3136008Z Metadata: read 2024-11-01T16:44:39.3136537Z Packages: read 2024-11-01T16:44:39.3136973Z Pages: read 2024-11-01T16:44:39.3137439Z PullRequests: read 2024-11-01T16:44:39.3137955Z RepositoryProjects: read 2024-11-01T16:44:39.3138507Z SecurityEvents: read 2024-11-01T16:44:39.3139037Z Statuses: read 2024-11-01T16:44:39.3139679Z ##[endgroup] 2024-11-01T16:44:39.3143466Z Secret source: Actions 2024-11-01T16:44:39.3144297Z Prepare workflow directory 2024-11-01T16:44:39.7188935Z Prepare all required actions 2024-11-01T16:44:39.7369265Z Getting action download info 2024-11-01T16:44:39.9305483Z Download action repository 'pytorch/test-infra@main' (SHA:49fb39b5efb49007791d74b09885044cd64544bf) 2024-11-01T16:44:40.2656377Z Download action repository 'pytorch/pytorch@main' (SHA:d79c5143d8e625947b93f8821bbe181419f47dee) 2024-11-01T16:44:43.2992176Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-11-01T16:44:43.4830776Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-11-01T16:44:43.7204994Z Getting action download info 2024-11-01T16:44:43.8272048Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-11-01T16:44:43.9795536Z Getting action download info 2024-11-01T16:44:44.1226951Z Download action repository 'nick-fields/retry@v3.0.0' (SHA:7152eba30c6575329ac0576536151aca5a72780e) 2024-11-01T16:44:44.2711801Z Getting action download info 2024-11-01T16:44:44.3688170Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-11-01T16:44:44.4973272Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/138766/merge (368da7510f5d755034681777a4e5ae6d33c07b38) 2024-11-01T16:44:44.4975802Z ##[group] Inputs 2024-11-01T16:44:44.4976331Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm75 2024-11-01T16:44:44.4977404Z test-matrix: {"include": [{"config": "pr_time_benchmarks", "shard": 1, "num_shards": 1, "runner": "linux.g4dn.metal.nvidia.gpu"}]} 2024-11-01T16:44:44.4979284Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:44:44.4980678Z sync-tag: 2024-11-01T16:44:44.4981438Z timeout-minutes: 240 2024-11-01T16:44:44.4981795Z use-gha: 2024-11-01T16:44:44.4982099Z dashboard-tag: 2024-11-01T16:44:44.4982445Z s3-bucket: gha-artifacts 2024-11-01T16:44:44.4982817Z aws-role-to-assume: 2024-11-01T16:44:44.4983188Z ##[endgroup] 2024-11-01T16:44:44.4984128Z Complete job name: cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) 2024-11-01T16:44:44.6054754Z A job started hook has been configured by the self-hosted runner administrator 2024-11-01T16:44:44.6200161Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-11-01T16:44:44.6208642Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:44:44.6209193Z ##[endgroup] 2024-11-01T16:44:45.9821165Z Runner Type: linux.g4dn.metal.nvidia.gpu 2024-11-01T16:44:45.9821716Z Instance Type: g4dn.metal 2024-11-01T16:44:45.9822057Z AMI Name: unknown 2024-11-01T16:44:45.9822592Z AMI ID: ami-0fff1b9a61dec8a5f 2024-11-01T16:44:51.4109074Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-11-01T16:44:51.4109971Z with: 2024-11-01T16:44:51.4110813Z github-secret: *** 2024-11-01T16:44:51.4111913Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-11-01T16:44:51.4113118Z activate-with-label: false 2024-11-01T16:44:51.4113540Z label: with-ssh 2024-11-01T16:44:51.4113911Z remove-existing-keys: true 2024-11-01T16:44:51.4114310Z fail-silently: true 2024-11-01T16:44:51.4114672Z env: 2024-11-01T16:44:51.4114988Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:44:51.4115391Z ##[endgroup] 2024-11-01T16:44:51.4972906Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-11-01T16:44:51.7727341Z Grabbing public ssh keys from https://github.com/c00w.keys 2024-11-01T16:44:51.8347482Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-11-01T16:44:51.8361960Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-11-01T16:44:51.8386978Z Login using: ssh ec2-user@ec2-54-209-19-37.compute-1.amazonaws.com 2024-11-01T16:44:51.8387927Z All testing is done inside the container, to start an interactive session run: 2024-11-01T16:44:51.8388908Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-11-01T16:44:51.8515572Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-11-01T16:44:51.8516191Z with: 2024-11-01T16:44:51.8516472Z submodules: recursive 2024-11-01T16:44:51.8516836Z fetch-depth: 0 2024-11-01T16:44:51.8517142Z env: 2024-11-01T16:44:51.8517423Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:44:51.8517800Z ##[endgroup] 2024-11-01T16:44:51.8716263Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2024-11-01T16:44:51.8717783Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2024-11-01T16:44:51.8724433Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:44:51.8724957Z env: 2024-11-01T16:44:51.8725229Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:44:51.8725575Z ##[endgroup] 2024-11-01T16:44:51.8811996Z ##[group]Run retry () { 2024-11-01T16:44:51.8812492Z retry () { 2024-11-01T16:44:51.8813084Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-11-01T16:44:51.8813726Z } 2024-11-01T16:44:51.8814096Z echo "${GITHUB_WORKSPACE}" 2024-11-01T16:44:51.8814598Z if [ -z "${NO_SUDO}" ]; then 2024-11-01T16:44:51.8815164Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-11-01T16:44:51.8815708Z else 2024-11-01T16:44:51.8816119Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-11-01T16:44:51.8816606Z fi 2024-11-01T16:44:51.8816976Z mkdir "${GITHUB_WORKSPACE}" 2024-11-01T16:44:51.8822347Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:44:51.8822885Z env: 2024-11-01T16:44:51.8823187Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:44:51.8823544Z NO_SUDO: 2024-11-01T16:44:51.8823915Z ##[endgroup] 2024-11-01T16:44:51.8845019Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-11-01T16:44:54.1051597Z ##[group]Run malfet/checkout@silent-checkout 2024-11-01T16:44:54.1052059Z with: 2024-11-01T16:44:54.1052412Z ref: d1aa4ef5a6c0ee2d57fb0086f5557e34537f0fea 2024-11-01T16:44:54.1052907Z fetch-depth: 0 2024-11-01T16:44:54.1053245Z submodules: recursive 2024-11-01T16:44:54.1053610Z quiet-checkout: true 2024-11-01T16:44:54.1053960Z repository: pytorch/pytorch 2024-11-01T16:44:54.1054486Z token: *** 2024-11-01T16:44:54.1054798Z ssh-strict: true 2024-11-01T16:44:54.1055145Z persist-credentials: true 2024-11-01T16:44:54.1055530Z clean: true 2024-11-01T16:44:54.1055851Z sparse-checkout-cone-mode: true 2024-11-01T16:44:54.1056273Z lfs: false 2024-11-01T16:44:54.1056798Z set-safe-directory: true 2024-11-01T16:44:54.1057149Z env: 2024-11-01T16:44:54.1057429Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:44:54.1057768Z ##[endgroup] 2024-11-01T16:44:54.2015680Z Syncing repository: pytorch/pytorch 2024-11-01T16:44:54.2017484Z ##[group]Getting Git version info 2024-11-01T16:44:54.2018373Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-11-01T16:44:54.2019349Z [command]/usr/bin/git version 2024-11-01T16:44:54.2019743Z git version 2.40.1 2024-11-01T16:44:54.2021255Z ##[endgroup] 2024-11-01T16:44:54.2033360Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/1a6c60db-6cbd-49f8-ba75-99beb5f811f1' before making global git config changes 2024-11-01T16:44:54.2034816Z Adding repository directory to the temporary git global config as a safe directory 2024-11-01T16:44:54.2036121Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-11-01T16:44:54.2067992Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-11-01T16:44:54.2071466Z ##[group]Initializing the repository 2024-11-01T16:44:54.2073862Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-11-01T16:44:54.2103146Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-11-01T16:44:54.2106346Z hint: is subject to change. To configure the initial branch name to use in all 2024-11-01T16:44:54.2108221Z hint: of your new repositories, which will suppress this warning, call: 2024-11-01T16:44:54.2108897Z hint: 2024-11-01T16:44:54.2109404Z hint: git config --global init.defaultBranch 2024-11-01T16:44:54.2110034Z hint: 2024-11-01T16:44:54.2110658Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-11-01T16:44:54.2111675Z hint: 'development'. The just-created branch can be renamed via this command: 2024-11-01T16:44:54.2112373Z hint: 2024-11-01T16:44:54.2112740Z hint: git branch -m 2024-11-01T16:44:54.2113623Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-11-01T16:44:54.2115364Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-11-01T16:44:54.2140385Z ##[endgroup] 2024-11-01T16:44:54.2142011Z ##[group]Disabling automatic garbage collection 2024-11-01T16:44:54.2143867Z [command]/usr/bin/git config --local gc.auto 0 2024-11-01T16:44:54.2170310Z ##[endgroup] 2024-11-01T16:44:54.2171800Z ##[group]Setting up auth 2024-11-01T16:44:54.2174024Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-11-01T16:44:54.2205199Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-11-01T16:44:54.2450073Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-11-01T16:44:54.2476612Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-11-01T16:44:54.2729843Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-11-01T16:44:54.2779798Z ##[endgroup] 2024-11-01T16:44:54.2781338Z ##[group]Fetching the repository 2024-11-01T16:44:54.2787584Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-11-01T16:44:57.0766229Z remote: Enumerating objects: 1053175 2024-11-01T16:44:57.0767905Z remote: Enumerating objects: 1053739, done. 2024-11-01T16:44:57.0769313Z remote: Counting objects: 0% (1/564) 2024-11-01T16:44:57.0769905Z remote: Counting objects: 1% (6/564) 2024-11-01T16:44:57.0770511Z remote: Counting objects: 2% (12/564) 2024-11-01T16:44:57.0771462Z remote: Counting objects: 3% (17/564) 2024-11-01T16:44:57.0772098Z remote: Counting objects: 4% (23/564) 2024-11-01T16:44:57.0772677Z remote: Counting objects: 5% (29/564) 2024-11-01T16:44:57.0773247Z remote: Counting objects: 6% (34/564) 2024-11-01T16:44:57.0773806Z remote: Counting objects: 7% (40/564) 2024-11-01T16:44:57.0774386Z remote: Counting objects: 8% (46/564) 2024-11-01T16:44:57.0774967Z remote: Counting objects: 9% (51/564) 2024-11-01T16:44:57.0775546Z remote: Counting objects: 10% (57/564) 2024-11-01T16:44:57.0776184Z remote: Counting objects: 11% (63/564) 2024-11-01T16:44:57.0776869Z remote: Counting objects: 12% (68/564) 2024-11-01T16:44:57.0777426Z remote: Counting objects: 13% (74/564) 2024-11-01T16:44:57.0777957Z remote: Counting objects: 14% (79/564) 2024-11-01T16:44:57.0778508Z remote: Counting objects: 15% (85/564) 2024-11-01T16:44:57.0779063Z remote: Counting objects: 16% (91/564) 2024-11-01T16:44:57.0779643Z remote: Counting objects: 17% (96/564) 2024-11-01T16:44:57.0780202Z remote: Counting objects: 18% (102/564) 2024-11-01T16:44:57.0780782Z remote: Counting objects: 19% (108/564) 2024-11-01T16:44:57.0781342Z remote: Counting objects: 20% (113/564) 2024-11-01T16:44:57.0782095Z remote: Counting objects: 21% (119/564) 2024-11-01T16:44:57.0782686Z remote: Counting objects: 22% (125/564) 2024-11-01T16:44:57.0783256Z remote: Counting objects: 23% (130/564) 2024-11-01T16:44:57.0783819Z remote: Counting objects: 24% (136/564) 2024-11-01T16:44:57.0784421Z remote: Counting objects: 25% (141/564) 2024-11-01T16:44:57.0784962Z remote: Counting objects: 26% (147/564) 2024-11-01T16:44:57.0785538Z remote: Counting objects: 27% (153/564) 2024-11-01T16:44:57.0786100Z remote: Counting objects: 28% (158/564) 2024-11-01T16:44:57.0786637Z remote: Counting objects: 29% (164/564) 2024-11-01T16:44:57.0787182Z remote: Counting objects: 30% (170/564) 2024-11-01T16:44:57.0787709Z remote: Counting objects: 31% (175/564) 2024-11-01T16:44:57.0788248Z remote: Counting objects: 32% (181/564) 2024-11-01T16:44:57.0788782Z remote: Counting objects: 33% (187/564) 2024-11-01T16:44:57.0789335Z remote: Counting objects: 34% (192/564) 2024-11-01T16:44:57.0789897Z remote: Counting objects: 35% (198/564) 2024-11-01T16:44:57.0790454Z remote: Counting objects: 36% (204/564) 2024-11-01T16:44:57.0791002Z remote: Counting objects: 37% (209/564) 2024-11-01T16:44:57.0791579Z remote: Counting objects: 38% (215/564) 2024-11-01T16:44:57.0792154Z remote: Counting objects: 39% (220/564) 2024-11-01T16:44:57.0792739Z remote: Counting objects: 40% (226/564) 2024-11-01T16:44:57.0793310Z remote: Counting objects: 41% (232/564) 2024-11-01T16:44:57.0793847Z remote: Counting objects: 42% (237/564) 2024-11-01T16:44:57.0794413Z remote: Counting objects: 43% (243/564) 2024-11-01T16:44:57.0794974Z remote: Counting objects: 44% (249/564) 2024-11-01T16:44:57.0795554Z remote: Counting objects: 45% (254/564) 2024-11-01T16:44:57.0796130Z remote: Counting objects: 46% (260/564) 2024-11-01T16:44:57.0796707Z remote: Counting objects: 47% (266/564) 2024-11-01T16:44:57.0797265Z remote: Counting objects: 48% (271/564) 2024-11-01T16:44:57.0797819Z remote: Counting objects: 49% (277/564) 2024-11-01T16:44:57.0798378Z remote: Counting objects: 50% (282/564) 2024-11-01T16:44:57.0798944Z remote: Counting objects: 51% (288/564) 2024-11-01T16:44:57.0799506Z remote: Counting objects: 52% (294/564) 2024-11-01T16:44:57.0800074Z remote: Counting objects: 53% (299/564) 2024-11-01T16:44:57.0800628Z remote: Counting objects: 54% (305/564) 2024-11-01T16:44:57.0801193Z remote: Counting objects: 55% (311/564) 2024-11-01T16:44:57.0801855Z remote: Counting objects: 56% (316/564) 2024-11-01T16:44:57.0802422Z remote: Counting objects: 57% (322/564) 2024-11-01T16:44:57.0803025Z remote: Counting objects: 58% (328/564) 2024-11-01T16:44:57.0803572Z remote: Counting objects: 59% (333/564) 2024-11-01T16:44:57.0804185Z remote: Counting objects: 60% (339/564) 2024-11-01T16:44:57.0804754Z remote: Counting objects: 61% (345/564) 2024-11-01T16:44:57.0805349Z remote: Counting objects: 62% (350/564) 2024-11-01T16:44:57.0805954Z remote: Counting objects: 63% (356/564) 2024-11-01T16:44:57.0806541Z remote: Counting objects: 64% (361/564) 2024-11-01T16:44:57.0807104Z remote: Counting objects: 65% (367/564) 2024-11-01T16:44:57.0807640Z remote: Counting objects: 66% (373/564) 2024-11-01T16:44:57.0808191Z remote: Counting objects: 67% (378/564) 2024-11-01T16:44:57.0808752Z remote: Counting objects: 68% (384/564) 2024-11-01T16:44:57.0809308Z remote: Counting objects: 69% (390/564) 2024-11-01T16:44:57.0810237Z remote: Counting objects: 70% (395/564) 2024-11-01T16:44:57.0810804Z remote: Counting objects: 71% (401/564) 2024-11-01T16:44:57.0811358Z remote: Counting objects: 72% (407/564) 2024-11-01T16:44:57.0812073Z remote: Counting objects: 73% (412/564) 2024-11-01T16:44:57.0812667Z remote: Counting objects: 74% (418/564) 2024-11-01T16:44:57.0813229Z remote: Counting objects: 75% (423/564) 2024-11-01T16:44:57.0813817Z remote: Counting objects: 76% (429/564) 2024-11-01T16:44:57.0814388Z remote: Counting objects: 77% (435/564) 2024-11-01T16:44:57.0814933Z remote: Counting objects: 78% (440/564) 2024-11-01T16:44:57.0815503Z remote: Counting objects: 79% (446/564) 2024-11-01T16:44:57.0816079Z remote: Counting objects: 80% (452/564) 2024-11-01T16:44:57.0816651Z remote: Counting objects: 81% (457/564) 2024-11-01T16:44:57.0817232Z remote: Counting objects: 82% (463/564) 2024-11-01T16:44:57.0817777Z remote: Counting objects: 83% (469/564) 2024-11-01T16:44:57.0818339Z remote: Counting objects: 84% (474/564) 2024-11-01T16:44:57.0818905Z remote: Counting objects: 85% (480/564) 2024-11-01T16:44:57.0819469Z remote: Counting objects: 86% (486/564) 2024-11-01T16:44:57.0820050Z remote: Counting objects: 87% (491/564) 2024-11-01T16:44:57.0820631Z remote: Counting objects: 88% (497/564) 2024-11-01T16:44:57.0821187Z remote: Counting objects: 89% (502/564) 2024-11-01T16:44:57.0821763Z remote: Counting objects: 90% (508/564) 2024-11-01T16:44:57.0822430Z remote: Counting objects: 91% (514/564) 2024-11-01T16:44:57.0822992Z remote: Counting objects: 92% (519/564) 2024-11-01T16:44:57.0823567Z remote: Counting objects: 93% (525/564) 2024-11-01T16:44:57.0824106Z remote: Counting objects: 94% (531/564) 2024-11-01T16:44:57.0824715Z remote: Counting objects: 95% (536/564) 2024-11-01T16:44:57.0825266Z remote: Counting objects: 96% (542/564) 2024-11-01T16:44:57.0825829Z remote: Counting objects: 97% (548/564) 2024-11-01T16:44:57.0826407Z remote: Counting objects: 98% (553/564) 2024-11-01T16:44:57.0826973Z remote: Counting objects: 99% (559/564) 2024-11-01T16:44:57.0827515Z remote: Counting objects: 100% (564/564) 2024-11-01T16:44:57.0828098Z remote: Counting objects: 100% (564/564), done. 2024-11-01T16:44:57.0828719Z remote: Compressing objects: 0% (1/314) 2024-11-01T16:44:57.0829505Z remote: Compressing objects: 1% (4/314) 2024-11-01T16:44:57.0960552Z remote: Compressing objects: 2% (7/314) 2024-11-01T16:44:57.1133023Z remote: Compressing objects: 3% (10/314) 2024-11-01T16:44:57.1266209Z remote: Compressing objects: 4% (13/314) 2024-11-01T16:44:57.1381379Z remote: Compressing objects: 5% (16/314) 2024-11-01T16:44:57.1534084Z remote: Compressing objects: 6% (19/314) 2024-11-01T16:44:57.1778292Z remote: Compressing objects: 7% (22/314) 2024-11-01T16:44:57.1877283Z remote: Compressing objects: 8% (26/314) 2024-11-01T16:44:57.1915356Z remote: Compressing objects: 9% (29/314) 2024-11-01T16:44:57.1916947Z remote: Compressing objects: 10% (32/314) 2024-11-01T16:44:57.1917550Z remote: Compressing objects: 11% (35/314) 2024-11-01T16:44:57.1918139Z remote: Compressing objects: 12% (38/314) 2024-11-01T16:44:57.1918716Z remote: Compressing objects: 13% (41/314) 2024-11-01T16:44:57.1919290Z remote: Compressing objects: 14% (44/314) 2024-11-01T16:44:57.1919965Z remote: Compressing objects: 15% (48/314) 2024-11-01T16:44:57.1920517Z remote: Compressing objects: 16% (51/314) 2024-11-01T16:44:57.1921068Z remote: Compressing objects: 17% (54/314) 2024-11-01T16:44:57.1921609Z remote: Compressing objects: 18% (57/314) 2024-11-01T16:44:57.1925464Z remote: Compressing objects: 19% (60/314) 2024-11-01T16:44:57.1928147Z remote: Compressing objects: 20% (63/314) 2024-11-01T16:44:57.1932565Z remote: Compressing objects: 21% (66/314) 2024-11-01T16:44:57.1934082Z remote: Compressing objects: 22% (70/314) 2024-11-01T16:44:57.1936056Z remote: Compressing objects: 23% (73/314) 2024-11-01T16:44:57.1937228Z remote: Compressing objects: 24% (76/314) 2024-11-01T16:44:57.1939079Z remote: Compressing objects: 25% (79/314) 2024-11-01T16:44:57.1940614Z remote: Compressing objects: 26% (82/314) 2024-11-01T16:44:57.1941374Z remote: Compressing objects: 27% (85/314) 2024-11-01T16:44:57.1942240Z remote: Compressing objects: 28% (88/314) 2024-11-01T16:44:57.1945512Z remote: Compressing objects: 29% (92/314) 2024-11-01T16:44:57.1946363Z remote: Compressing objects: 30% (95/314) 2024-11-01T16:44:57.1947356Z remote: Compressing objects: 31% (98/314) 2024-11-01T16:44:57.1949317Z remote: Compressing objects: 32% (101/314) 2024-11-01T16:44:57.1951616Z remote: Compressing objects: 33% (104/314) 2024-11-01T16:44:57.1958636Z remote: Compressing objects: 34% (107/314) 2024-11-01T16:44:57.1962535Z remote: Compressing objects: 35% (110/314) 2024-11-01T16:44:57.1963297Z remote: Compressing objects: 36% (114/314) 2024-11-01T16:44:57.1964965Z remote: Compressing objects: 37% (117/314) 2024-11-01T16:44:57.1970691Z remote: Compressing objects: 38% (120/314) 2024-11-01T16:44:57.1979579Z remote: Compressing objects: 39% (123/314) 2024-11-01T16:44:57.1982105Z remote: Compressing objects: 40% (126/314) 2024-11-01T16:44:57.1984385Z remote: Compressing objects: 41% (129/314) 2024-11-01T16:44:57.1991080Z remote: Compressing objects: 42% (132/314) 2024-11-01T16:44:57.1994919Z remote: Compressing objects: 43% (136/314) 2024-11-01T16:44:57.1995656Z remote: Compressing objects: 44% (139/314) 2024-11-01T16:44:57.2001327Z remote: Compressing objects: 45% (142/314) 2024-11-01T16:44:57.2003448Z remote: Compressing objects: 46% (145/314) 2024-11-01T16:44:57.2006846Z remote: Compressing objects: 47% (148/314) 2024-11-01T16:44:57.2007471Z remote: Compressing objects: 48% (151/314) 2024-11-01T16:44:57.2010324Z remote: Compressing objects: 49% (154/314) 2024-11-01T16:44:57.2012791Z remote: Compressing objects: 50% (157/314) 2024-11-01T16:44:57.2014445Z remote: Compressing objects: 51% (161/314) 2024-11-01T16:44:57.2016017Z remote: Compressing objects: 52% (164/314) 2024-11-01T16:44:57.2018329Z remote: Compressing objects: 53% (167/314) 2024-11-01T16:44:57.2020544Z remote: Compressing objects: 54% (170/314) 2024-11-01T16:44:57.2024610Z remote: Compressing objects: 55% (173/314) 2024-11-01T16:44:57.2026644Z remote: Compressing objects: 56% (176/314) 2024-11-01T16:44:57.2028734Z remote: Compressing objects: 57% (179/314) 2024-11-01T16:44:57.2029817Z remote: Compressing objects: 58% (183/314) 2024-11-01T16:44:57.2030463Z remote: Compressing objects: 59% (186/314) 2024-11-01T16:44:57.2031074Z remote: Compressing objects: 60% (189/314) 2024-11-01T16:44:57.2031718Z remote: Compressing objects: 61% (192/314) 2024-11-01T16:44:57.2032340Z remote: Compressing objects: 62% (195/314) 2024-11-01T16:44:57.2032965Z remote: Compressing objects: 63% (198/314) 2024-11-01T16:44:57.2033563Z remote: Compressing objects: 64% (201/314) 2024-11-01T16:44:57.2034200Z remote: Compressing objects: 65% (205/314) 2024-11-01T16:44:57.2034799Z remote: Compressing objects: 66% (208/314) 2024-11-01T16:44:57.2035439Z remote: Compressing objects: 67% (211/314) 2024-11-01T16:44:57.2036067Z remote: Compressing objects: 68% (214/314) 2024-11-01T16:44:57.2036689Z remote: Compressing objects: 69% (217/314) 2024-11-01T16:44:57.2037325Z remote: Compressing objects: 70% (220/314) 2024-11-01T16:44:57.2037958Z remote: Compressing objects: 71% (223/314) 2024-11-01T16:44:57.2038602Z remote: Compressing objects: 72% (227/314) 2024-11-01T16:44:57.2039230Z remote: Compressing objects: 73% (230/314) 2024-11-01T16:44:57.2040048Z remote: Compressing objects: 74% (233/314) 2024-11-01T16:44:57.2040717Z remote: Compressing objects: 75% (236/314) 2024-11-01T16:44:57.2041321Z remote: Compressing objects: 76% (239/314) 2024-11-01T16:44:57.2041953Z remote: Compressing objects: 77% (242/314) 2024-11-01T16:44:57.2043237Z remote: Compressing objects: 78% (245/314) 2024-11-01T16:44:57.2044696Z remote: Compressing objects: 79% (249/314) 2024-11-01T16:44:57.2045964Z remote: Compressing objects: 80% (252/314) 2024-11-01T16:44:57.2047193Z remote: Compressing objects: 81% (255/314) 2024-11-01T16:44:57.2048426Z remote: Compressing objects: 82% (258/314) 2024-11-01T16:44:57.2049572Z remote: Compressing objects: 83% (261/314) 2024-11-01T16:44:57.2050789Z remote: Compressing objects: 84% (264/314) 2024-11-01T16:44:57.2052660Z remote: Compressing objects: 85% (267/314) 2024-11-01T16:44:57.2053845Z remote: Compressing objects: 86% (271/314) 2024-11-01T16:44:57.2054509Z remote: Compressing objects: 87% (274/314) 2024-11-01T16:44:57.2056022Z remote: Compressing objects: 88% (277/314) 2024-11-01T16:44:57.2057580Z remote: Compressing objects: 89% (280/314) 2024-11-01T16:44:57.2058740Z remote: Compressing objects: 90% (283/314) 2024-11-01T16:44:57.2059646Z remote: Compressing objects: 91% (286/314) 2024-11-01T16:44:57.2060617Z remote: Compressing objects: 92% (289/314) 2024-11-01T16:44:57.2061797Z remote: Compressing objects: 93% (293/314) 2024-11-01T16:44:57.2062527Z remote: Compressing objects: 94% (296/314) 2024-11-01T16:44:57.2063318Z remote: Compressing objects: 95% (299/314) 2024-11-01T16:44:57.2064128Z remote: Compressing objects: 96% (302/314) 2024-11-01T16:44:57.2064776Z remote: Compressing objects: 97% (305/314) 2024-11-01T16:44:57.2065463Z remote: Compressing objects: 98% (308/314) 2024-11-01T16:44:57.2066228Z remote: Compressing objects: 99% (311/314) 2024-11-01T16:44:57.2066866Z remote: Compressing objects: 100% (314/314) 2024-11-01T16:44:57.2067445Z remote: Compressing objects: 100% (314/314), done. 2024-11-01T16:45:18.7631555Z remote: Total 1053739 (delta 348), reused 416 (delta 249), pack-reused 1053175 (from 1) 2024-11-01T16:45:40.4025681Z [command]/usr/bin/git rev-parse --verify --quiet d1aa4ef5a6c0ee2d57fb0086f5557e34537f0fea^{object} 2024-11-01T16:45:40.4054733Z d1aa4ef5a6c0ee2d57fb0086f5557e34537f0fea 2024-11-01T16:45:40.4059030Z ##[endgroup] 2024-11-01T16:45:40.4059661Z ##[group]Determining the checkout info 2024-11-01T16:45:40.4060620Z ##[endgroup] 2024-11-01T16:45:40.4061333Z ##[group]Checking out the ref 2024-11-01T16:45:40.4062518Z [command]/usr/bin/git checkout --quiet --force d1aa4ef5a6c0ee2d57fb0086f5557e34537f0fea 2024-11-01T16:45:41.7787046Z ##[endgroup] 2024-11-01T16:45:41.7789040Z ##[group]Setting up auth for fetching submodules 2024-11-01T16:45:41.7790930Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-11-01T16:45:41.7855227Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-11-01T16:45:41.7883731Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-11-01T16:45:41.7910864Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-11-01T16:45:41.7937223Z ##[endgroup] 2024-11-01T16:45:41.7938686Z ##[group]Fetching submodules 2024-11-01T16:45:41.7939946Z [command]/usr/bin/git submodule sync --recursive 2024-11-01T16:45:41.8226777Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-11-01T16:45:41.8493656Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-11-01T16:45:41.8498182Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-11-01T16:45:41.8499840Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-11-01T16:45:41.8501429Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-11-01T16:45:41.8502870Z Submodule 'third_party/NVTX' (https://github.com/NVIDIA/NVTX.git) registered for path 'third_party/NVTX' 2024-11-01T16:45:41.8504753Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-11-01T16:45:41.8506680Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-11-01T16:45:41.8508215Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-11-01T16:45:41.8510180Z Submodule 'third_party/composable_kernel' (https://github.com/ROCm/composable_kernel.git) registered for path 'third_party/composable_kernel' 2024-11-01T16:45:41.8511994Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-11-01T16:45:41.8513628Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-11-01T16:45:41.8515332Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-11-01T16:45:41.8517049Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-11-01T16:45:41.8519822Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-11-01T16:45:41.8522811Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-11-01T16:45:41.8525742Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-11-01T16:45:41.8531258Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-11-01T16:45:41.8535421Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-11-01T16:45:41.8537056Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-11-01T16:45:41.8539495Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-11-01T16:45:41.8542533Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-11-01T16:45:41.8545851Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-11-01T16:45:41.8549126Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-11-01T16:45:41.8552546Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-11-01T16:45:41.8557685Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-11-01T16:45:41.8560796Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-11-01T16:45:41.8563411Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-11-01T16:45:41.8567452Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-11-01T16:45:41.8572294Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-11-01T16:45:41.8576877Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-11-01T16:45:41.8579112Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-11-01T16:45:41.8582796Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-11-01T16:45:41.8586622Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-11-01T16:45:41.8593706Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-11-01T16:45:41.8597589Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-11-01T16:45:41.8600638Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-11-01T16:45:41.8627349Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-11-01T16:45:42.2448704Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-11-01T16:45:42.4527107Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-11-01T16:45:42.7058526Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-11-01T16:45:43.7143164Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NVTX'... 2024-11-01T16:45:44.0287008Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-11-01T16:45:45.9961448Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-11-01T16:45:55.4665315Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-11-01T16:45:55.9027006Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/composable_kernel'... 2024-11-01T16:45:57.6121177Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-11-01T16:45:58.1114572Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-11-01T16:45:58.7139855Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-11-01T16:46:00.0024656Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-11-01T16:46:02.2629813Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-11-01T16:46:08.4547664Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-11-01T16:46:09.9991201Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-11-01T16:46:11.7165354Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-11-01T16:46:12.8333405Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-11-01T16:46:13.2962534Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-11-01T16:46:13.6778167Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-11-01T16:46:14.6857104Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-11-01T16:46:15.0858669Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-11-01T16:46:15.3445539Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-11-01T16:46:16.9605529Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-11-01T16:46:17.7124259Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-11-01T16:46:18.1291274Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-11-01T16:46:23.8551822Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-11-01T16:46:25.7674897Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-11-01T16:46:32.3477057Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-11-01T16:46:32.6007367Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-11-01T16:46:41.8447557Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-11-01T16:46:42.0396470Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-11-01T16:46:42.2502760Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-11-01T16:46:43.2392972Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-11-01T16:46:43.5378646Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-11-01T16:46:44.2339790Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-11-01T16:46:44.6963327Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-11-01T16:46:44.7067212Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-11-01T16:46:44.7161130Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-11-01T16:46:44.7381711Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-11-01T16:46:44.7671308Z Submodule path 'third_party/NVTX': checked out 'e170594ac7cf1dac584da473d4ca9301087090c1' 2024-11-01T16:46:44.8041104Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-11-01T16:46:45.5557345Z Submodule path 'third_party/XNNPACK': checked out '87ee0b46b834f67bad9025d4a82ed5654f3403d3' 2024-11-01T16:46:45.5783668Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-11-01T16:46:45.7753915Z Submodule path 'third_party/composable_kernel': checked out 'cedccd59c94cb0c74e7ec0d0f6c791aed081febc' 2024-11-01T16:46:45.8203939Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-11-01T16:46:45.9137811Z Submodule path 'third_party/cpuinfo': checked out '1e83a2fdd3102f65c6f1fb602c1b320486218a99' 2024-11-01T16:46:45.9455144Z Submodule path 'third_party/cudnn_frontend': checked out '936021bfed8c91dc416af1588b2c4eca631a9e45' 2024-11-01T16:46:46.4300379Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-11-01T16:46:46.6740280Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-11-01T16:46:46.7607843Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-11-01T16:46:46.7626034Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-11-01T16:46:46.7630575Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-11-01T16:46:46.7632628Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-11-01T16:46:46.7634468Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-11-01T16:46:46.7636530Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-11-01T16:46:46.7659863Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-11-01T16:46:47.8762392Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-11-01T16:46:48.5060383Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-11-01T16:46:50.6777077Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-11-01T16:46:51.5942829Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-11-01T16:46:52.0005576Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-11-01T16:46:52.0932764Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-11-01T16:46:52.4754541Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-11-01T16:46:52.5353386Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-11-01T16:46:52.5470315Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-11-01T16:46:52.6564077Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-11-01T16:46:52.6928892Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-11-01T16:46:52.7293699Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-11-01T16:46:52.7533363Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-11-01T16:46:52.7954759Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-11-01T16:46:52.8071993Z Submodule path 'third_party/ideep': checked out '41d636c2bbcea6bff0faf97cdb65a48cdde987af' 2024-11-01T16:46:52.8089080Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-11-01T16:46:52.8111516Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-11-01T16:47:04.9619465Z Submodule path 'third_party/ideep/mkl-dnn': checked out '66f0cb9eb66affd2da3bf5f8d897376f04aae6af' 2024-11-01T16:47:04.9794376Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-11-01T16:47:05.0645417Z Submodule path 'third_party/kineto': checked out 'ed052ea024b9468908d558b15cd3f7584fb0f492' 2024-11-01T16:47:05.0667207Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-11-01T16:47:05.0673256Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-11-01T16:47:05.0675564Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-11-01T16:47:05.0690893Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-11-01T16:47:05.6047704Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-11-01T16:47:06.6603504Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-11-01T16:47:07.6672231Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-11-01T16:47:07.6688705Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-11-01T16:47:07.6691933Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-11-01T16:47:07.6694046Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-11-01T16:47:07.6696169Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-11-01T16:47:07.6698288Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-11-01T16:47:07.6700525Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-11-01T16:47:07.6702757Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-11-01T16:47:07.6704848Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-11-01T16:47:07.6728834Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-11-01T16:47:08.5693670Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-11-01T16:47:09.0049720Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-11-01T16:47:10.0805899Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-11-01T16:47:10.3807474Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-11-01T16:47:10.9413814Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-11-01T16:47:11.8868408Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-11-01T16:47:18.3519563Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-11-01T16:47:18.9722385Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-11-01T16:47:18.9895937Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-11-01T16:47:19.0237865Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-11-01T16:47:19.0357264Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-11-01T16:47:19.0370072Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-11-01T16:47:19.0393645Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-11-01T16:47:19.4242550Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-11-01T16:47:19.4406312Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-11-01T16:47:19.4782775Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-11-01T16:47:19.5738299Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-11-01T16:47:19.5893776Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-11-01T16:47:19.6248611Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-11-01T16:47:19.6789560Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-11-01T16:47:19.7155796Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-11-01T16:47:19.7436847Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-11-01T16:47:19.8415096Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-11-01T16:47:20.1687361Z Submodule path 'third_party/onnx': checked out 'b8baa8446686496da4cc8fda09f2b6fe65c2a02c' 2024-11-01T16:47:20.1721750Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-11-01T16:47:20.1743716Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-11-01T16:47:21.2786687Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-11-01T16:47:21.3437495Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-11-01T16:47:21.3457213Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-11-01T16:47:21.3459194Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-11-01T16:47:21.3461065Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-11-01T16:47:21.3462979Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-11-01T16:47:21.3465198Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-11-01T16:47:21.3467575Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-11-01T16:47:21.3470086Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-11-01T16:47:21.3472122Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-11-01T16:47:21.3495928Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-11-01T16:47:21.7703125Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-11-01T16:47:22.7686360Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-11-01T16:47:23.0868678Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-11-01T16:47:28.6881441Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-11-01T16:47:28.9914433Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-11-01T16:47:29.2354882Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-11-01T16:47:29.5664301Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-11-01T16:47:37.3619469Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-11-01T16:47:37.4006650Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-11-01T16:47:37.4152658Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-11-01T16:47:37.5094153Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-11-01T16:47:37.5212267Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-11-01T16:47:37.5342107Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-11-01T16:47:37.5484045Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-11-01T16:47:37.5498364Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-11-01T16:47:37.5500711Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-11-01T16:47:37.5521168Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-11-01T16:47:39.2730424Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-11-01T16:47:40.4823224Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-11-01T16:47:40.5253604Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-11-01T16:47:40.9457500Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-11-01T16:47:40.9574317Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-11-01T16:47:41.2363638Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-11-01T16:47:41.2385020Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-11-01T16:47:41.2387089Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-11-01T16:47:41.2409750Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-11-01T16:47:41.7215775Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-11-01T16:47:42.6763089Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-11-01T16:47:42.7438391Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-11-01T16:47:42.7520236Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-11-01T16:47:42.7639958Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-11-01T16:47:42.7971860Z Submodule path 'third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4' 2024-11-01T16:47:42.8239873Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-11-01T16:47:42.8632665Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-11-01T16:47:42.8881609Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-11-01T16:47:42.8897394Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-11-01T16:47:42.8899462Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-11-01T16:47:42.8901343Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-11-01T16:47:42.8903239Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-11-01T16:47:42.8926095Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-11-01T16:47:43.9218688Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-11-01T16:47:44.1771948Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-11-01T16:47:45.5704488Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-11-01T16:47:47.1854620Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-11-01T16:47:47.1990721Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-11-01T16:47:47.2642874Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-11-01T16:47:47.2905099Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-11-01T16:47:47.2920931Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-11-01T16:47:47.2942851Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-11-01T16:47:47.5235435Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-11-01T16:47:47.5277038Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-11-01T16:47:47.5549851Z Entering 'android/libs/fbjni' 2024-11-01T16:47:47.5584813Z Entering 'third_party/FP16' 2024-11-01T16:47:47.5627767Z Entering 'third_party/FXdiv' 2024-11-01T16:47:47.5664036Z Entering 'third_party/NNPACK' 2024-11-01T16:47:47.5708489Z Entering 'third_party/NVTX' 2024-11-01T16:47:47.5755749Z Entering 'third_party/VulkanMemoryAllocator' 2024-11-01T16:47:47.5797820Z Entering 'third_party/XNNPACK' 2024-11-01T16:47:47.5859310Z Entering 'third_party/benchmark' 2024-11-01T16:47:47.5896294Z Entering 'third_party/composable_kernel' 2024-11-01T16:47:47.5949847Z Entering 'third_party/cpp-httplib' 2024-11-01T16:47:47.5991737Z Entering 'third_party/cpuinfo' 2024-11-01T16:47:47.6036295Z Entering 'third_party/cudnn_frontend' 2024-11-01T16:47:47.6086284Z Entering 'third_party/cutlass' 2024-11-01T16:47:47.6130319Z Entering 'third_party/eigen' 2024-11-01T16:47:47.6180059Z Entering 'third_party/fbgemm' 2024-11-01T16:47:47.6220595Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-11-01T16:47:47.6264255Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-11-01T16:47:47.6306805Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-11-01T16:47:47.6350769Z Entering 'third_party/fbgemm/third_party/googletest' 2024-11-01T16:47:47.6394155Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-11-01T16:47:47.6436835Z Entering 'third_party/flatbuffers' 2024-11-01T16:47:47.6480161Z Entering 'third_party/fmt' 2024-11-01T16:47:47.6521937Z Entering 'third_party/gemmlowp/gemmlowp' 2024-11-01T16:47:47.6566000Z Entering 'third_party/gloo' 2024-11-01T16:47:47.6606334Z Entering 'third_party/googletest' 2024-11-01T16:47:47.6648359Z Entering 'third_party/ideep' 2024-11-01T16:47:47.6693569Z Entering 'third_party/ideep/mkl-dnn' 2024-11-01T16:47:47.6740577Z Entering 'third_party/ittapi' 2024-11-01T16:47:47.6782470Z Entering 'third_party/kineto' 2024-11-01T16:47:47.6828665Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-11-01T16:47:47.6876074Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-11-01T16:47:47.6911947Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-11-01T16:47:47.6958419Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-11-01T16:47:47.6998027Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-11-01T16:47:47.7032147Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-11-01T16:47:47.7069799Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-11-01T16:47:47.7107935Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-11-01T16:47:47.7150203Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-11-01T16:47:47.7186422Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-11-01T16:47:47.7221795Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-11-01T16:47:47.7261144Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-11-01T16:47:47.7299392Z Entering 'third_party/mimalloc' 2024-11-01T16:47:47.7343082Z Entering 'third_party/nccl/nccl' 2024-11-01T16:47:47.7380765Z Entering 'third_party/nlohmann' 2024-11-01T16:47:47.7428256Z Entering 'third_party/onnx' 2024-11-01T16:47:47.7481268Z Entering 'third_party/onnx/third_party/pybind11' 2024-11-01T16:47:47.7529736Z Entering 'third_party/opentelemetry-cpp' 2024-11-01T16:47:47.7567361Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-11-01T16:47:47.7609858Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-11-01T16:47:47.7649840Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-11-01T16:47:47.7684514Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-11-01T16:47:47.7719887Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-11-01T16:47:47.7761848Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-11-01T16:47:47.7796967Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-11-01T16:47:47.7832650Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-11-01T16:47:47.7872994Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-11-01T16:47:47.7911402Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-11-01T16:47:47.7969263Z Entering 'third_party/pocketfft' 2024-11-01T16:47:47.8011145Z Entering 'third_party/protobuf' 2024-11-01T16:47:47.8059914Z Entering 'third_party/protobuf/third_party/benchmark' 2024-11-01T16:47:47.8094690Z Entering 'third_party/protobuf/third_party/googletest' 2024-11-01T16:47:47.8134786Z Entering 'third_party/psimd' 2024-11-01T16:47:47.8175658Z Entering 'third_party/pthreadpool' 2024-11-01T16:47:47.8216486Z Entering 'third_party/pybind11' 2024-11-01T16:47:47.8260086Z Entering 'third_party/python-peachpy' 2024-11-01T16:47:47.8301125Z Entering 'third_party/sleef' 2024-11-01T16:47:47.8346403Z Entering 'third_party/tensorpipe' 2024-11-01T16:47:47.8387109Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-11-01T16:47:47.8423480Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-11-01T16:47:47.8464167Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-11-01T16:47:47.8506775Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-11-01T16:47:47.8551204Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-11-01T16:47:47.8609719Z ##[endgroup] 2024-11-01T16:47:47.8610737Z ##[group]Persisting credentials for submodules 2024-11-01T16:47:47.8613311Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-11-01T16:47:47.8879698Z Entering 'android/libs/fbjni' 2024-11-01T16:47:47.8930139Z Entering 'third_party/FP16' 2024-11-01T16:47:47.8977210Z Entering 'third_party/FXdiv' 2024-11-01T16:47:47.9031504Z Entering 'third_party/NNPACK' 2024-11-01T16:47:47.9077958Z Entering 'third_party/NVTX' 2024-11-01T16:47:47.9131773Z Entering 'third_party/VulkanMemoryAllocator' 2024-11-01T16:47:47.9185973Z Entering 'third_party/XNNPACK' 2024-11-01T16:47:47.9246486Z Entering 'third_party/benchmark' 2024-11-01T16:47:47.9295987Z Entering 'third_party/composable_kernel' 2024-11-01T16:47:47.9349338Z Entering 'third_party/cpp-httplib' 2024-11-01T16:47:47.9405866Z Entering 'third_party/cpuinfo' 2024-11-01T16:47:47.9467119Z Entering 'third_party/cudnn_frontend' 2024-11-01T16:47:47.9528013Z Entering 'third_party/cutlass' 2024-11-01T16:47:47.9586769Z Entering 'third_party/eigen' 2024-11-01T16:47:47.9634914Z Entering 'third_party/fbgemm' 2024-11-01T16:47:47.9695714Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-11-01T16:47:47.9745920Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-11-01T16:47:47.9805132Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-11-01T16:47:47.9865763Z Entering 'third_party/fbgemm/third_party/googletest' 2024-11-01T16:47:47.9916137Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-11-01T16:47:47.9963478Z Entering 'third_party/flatbuffers' 2024-11-01T16:47:48.0020162Z Entering 'third_party/fmt' 2024-11-01T16:47:48.0076764Z Entering 'third_party/gemmlowp/gemmlowp' 2024-11-01T16:47:48.0130488Z Entering 'third_party/gloo' 2024-11-01T16:47:48.0186336Z Entering 'third_party/googletest' 2024-11-01T16:47:48.0235725Z Entering 'third_party/ideep' 2024-11-01T16:47:48.0286188Z Entering 'third_party/ideep/mkl-dnn' 2024-11-01T16:47:48.0340778Z Entering 'third_party/ittapi' 2024-11-01T16:47:48.0387783Z Entering 'third_party/kineto' 2024-11-01T16:47:48.0446269Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-11-01T16:47:48.0494086Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-11-01T16:47:48.0545014Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-11-01T16:47:48.0595171Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-11-01T16:47:48.0642243Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-11-01T16:47:48.0687305Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-11-01T16:47:48.0749707Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-11-01T16:47:48.0796132Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-11-01T16:47:48.0844513Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-11-01T16:47:48.0896578Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-11-01T16:47:48.0946085Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-11-01T16:47:48.0991068Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-11-01T16:47:48.1039038Z Entering 'third_party/mimalloc' 2024-11-01T16:47:48.1097051Z Entering 'third_party/nccl/nccl' 2024-11-01T16:47:48.1148344Z Entering 'third_party/nlohmann' 2024-11-01T16:47:48.1202952Z Entering 'third_party/onnx' 2024-11-01T16:47:48.1268492Z Entering 'third_party/onnx/third_party/pybind11' 2024-11-01T16:47:48.1329209Z Entering 'third_party/opentelemetry-cpp' 2024-11-01T16:47:48.1382092Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-11-01T16:47:48.1427775Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-11-01T16:47:48.1472059Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-11-01T16:47:48.1517097Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-11-01T16:47:48.1577841Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-11-01T16:47:48.1630325Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-11-01T16:47:48.1683643Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-11-01T16:47:48.1735821Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-11-01T16:47:48.1791110Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-11-01T16:47:48.1840911Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-11-01T16:47:48.1905295Z Entering 'third_party/pocketfft' 2024-11-01T16:47:48.1956670Z Entering 'third_party/protobuf' 2024-11-01T16:47:48.2010837Z Entering 'third_party/protobuf/third_party/benchmark' 2024-11-01T16:47:48.2066368Z Entering 'third_party/protobuf/third_party/googletest' 2024-11-01T16:47:48.2126609Z Entering 'third_party/psimd' 2024-11-01T16:47:48.2177775Z Entering 'third_party/pthreadpool' 2024-11-01T16:47:48.2224993Z Entering 'third_party/pybind11' 2024-11-01T16:47:48.2271459Z Entering 'third_party/python-peachpy' 2024-11-01T16:47:48.2326765Z Entering 'third_party/sleef' 2024-11-01T16:47:48.2386303Z Entering 'third_party/tensorpipe' 2024-11-01T16:47:48.2446193Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-11-01T16:47:48.2490670Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-11-01T16:47:48.2537382Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-11-01T16:47:48.2591572Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-11-01T16:47:48.2640594Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-11-01T16:47:48.2702176Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-11-01T16:47:48.2971150Z Entering 'android/libs/fbjni' 2024-11-01T16:47:48.3030399Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-11-01T16:47:48.3039468Z Entering 'third_party/FP16' 2024-11-01T16:47:48.3084546Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-11-01T16:47:48.3101211Z Entering 'third_party/FXdiv' 2024-11-01T16:47:48.3145750Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-11-01T16:47:48.3166969Z Entering 'third_party/NNPACK' 2024-11-01T16:47:48.3213707Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-11-01T16:47:48.3231015Z Entering 'third_party/NVTX' 2024-11-01T16:47:48.3278651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config remote.origin.url 2024-11-01T16:47:48.3298556Z Entering 'third_party/VulkanMemoryAllocator' 2024-11-01T16:47:48.3346000Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-11-01T16:47:48.3362395Z Entering 'third_party/XNNPACK' 2024-11-01T16:47:48.3406639Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-11-01T16:47:48.3440125Z Entering 'third_party/benchmark' 2024-11-01T16:47:48.3482259Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-11-01T16:47:48.3497653Z Entering 'third_party/composable_kernel' 2024-11-01T16:47:48.3547397Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config remote.origin.url 2024-11-01T16:47:48.3572929Z Entering 'third_party/cpp-httplib' 2024-11-01T16:47:48.3621454Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-11-01T16:47:48.3640853Z Entering 'third_party/cpuinfo' 2024-11-01T16:47:48.3686692Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-11-01T16:47:48.3706900Z Entering 'third_party/cudnn_frontend' 2024-11-01T16:47:48.3752849Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-11-01T16:47:48.3769286Z Entering 'third_party/cutlass' 2024-11-01T16:47:48.3815643Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-11-01T16:47:48.3839319Z Entering 'third_party/eigen' 2024-11-01T16:47:48.3883037Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-11-01T16:47:48.3900080Z Entering 'third_party/fbgemm' 2024-11-01T16:47:48.3949794Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-11-01T16:47:48.3966251Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-11-01T16:47:48.4007308Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-11-01T16:47:48.4026400Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-11-01T16:47:48.4068254Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-11-01T16:47:48.4087891Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-11-01T16:47:48.4134958Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-11-01T16:47:48.4155905Z Entering 'third_party/fbgemm/third_party/googletest' 2024-11-01T16:47:48.4201941Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-11-01T16:47:48.4215809Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-11-01T16:47:48.4259947Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-11-01T16:47:48.4284397Z Entering 'third_party/flatbuffers' 2024-11-01T16:47:48.4328449Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-11-01T16:47:48.4346809Z Entering 'third_party/fmt' 2024-11-01T16:47:48.4394729Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-11-01T16:47:48.4410810Z Entering 'third_party/gemmlowp/gemmlowp' 2024-11-01T16:47:48.4456905Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-11-01T16:47:48.4473931Z Entering 'third_party/gloo' 2024-11-01T16:47:48.4518242Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-11-01T16:47:48.4536240Z Entering 'third_party/googletest' 2024-11-01T16:47:48.4584708Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-11-01T16:47:48.4601341Z Entering 'third_party/ideep' 2024-11-01T16:47:48.4647232Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-11-01T16:47:48.4667269Z Entering 'third_party/ideep/mkl-dnn' 2024-11-01T16:47:48.4715092Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-11-01T16:47:48.4736204Z Entering 'third_party/ittapi' 2024-11-01T16:47:48.4781528Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-11-01T16:47:48.4798448Z Entering 'third_party/kineto' 2024-11-01T16:47:48.4845829Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-11-01T16:47:48.4861782Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-11-01T16:47:48.4905233Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-11-01T16:47:48.4920002Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-11-01T16:47:48.4964297Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-11-01T16:47:48.4980518Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-11-01T16:47:48.5024469Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-11-01T16:47:48.5036271Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-11-01T16:47:48.5075836Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-11-01T16:47:48.5096786Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-11-01T16:47:48.5141684Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-11-01T16:47:48.5151876Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-11-01T16:47:48.5194005Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-11-01T16:47:48.5207321Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-11-01T16:47:48.5256192Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-11-01T16:47:48.5269476Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-11-01T16:47:48.5313414Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-11-01T16:47:48.5325622Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-11-01T16:47:48.5368003Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-11-01T16:47:48.5380366Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-11-01T16:47:48.5423148Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-11-01T16:47:48.5438804Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-11-01T16:47:48.5485651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-11-01T16:47:48.5505675Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-11-01T16:47:48.5555230Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-11-01T16:47:48.5582434Z Entering 'third_party/mimalloc' 2024-11-01T16:47:48.5628575Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-11-01T16:47:48.5648552Z Entering 'third_party/nccl/nccl' 2024-11-01T16:47:48.5692605Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-11-01T16:47:48.5709883Z Entering 'third_party/nlohmann' 2024-11-01T16:47:48.5753754Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-11-01T16:47:48.5778539Z Entering 'third_party/onnx' 2024-11-01T16:47:48.5826531Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-11-01T16:47:48.5854061Z Entering 'third_party/onnx/third_party/pybind11' 2024-11-01T16:47:48.5896089Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-11-01T16:47:48.5915571Z Entering 'third_party/opentelemetry-cpp' 2024-11-01T16:47:48.5962082Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-11-01T16:47:48.5979015Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-11-01T16:47:48.6020556Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-11-01T16:47:48.6035761Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-11-01T16:47:48.6077129Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-11-01T16:47:48.6097948Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-11-01T16:47:48.6146324Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-11-01T16:47:48.6162451Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-11-01T16:47:48.6205077Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-11-01T16:47:48.6217220Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-11-01T16:47:48.6261475Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-11-01T16:47:48.6279174Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-11-01T16:47:48.6325863Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-11-01T16:47:48.6338591Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-11-01T16:47:48.6386766Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-11-01T16:47:48.6398927Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-11-01T16:47:48.6439116Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-11-01T16:47:48.6453531Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-11-01T16:47:48.6496085Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-11-01T16:47:48.6512081Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-11-01T16:47:48.6554712Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-11-01T16:47:48.6587086Z Entering 'third_party/pocketfft' 2024-11-01T16:47:48.6637537Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-11-01T16:47:48.6653172Z Entering 'third_party/protobuf' 2024-11-01T16:47:48.6695763Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-11-01T16:47:48.6720602Z Entering 'third_party/protobuf/third_party/benchmark' 2024-11-01T16:47:48.6760983Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-11-01T16:47:48.6772108Z Entering 'third_party/protobuf/third_party/googletest' 2024-11-01T16:47:48.6813686Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-11-01T16:47:48.6827169Z Entering 'third_party/psimd' 2024-11-01T16:47:48.6871816Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-11-01T16:47:48.6891855Z Entering 'third_party/pthreadpool' 2024-11-01T16:47:48.6941224Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-11-01T16:47:48.6958462Z Entering 'third_party/pybind11' 2024-11-01T16:47:48.7004879Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-11-01T16:47:48.7020701Z Entering 'third_party/python-peachpy' 2024-11-01T16:47:48.7062698Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-11-01T16:47:48.7078782Z Entering 'third_party/sleef' 2024-11-01T16:47:48.7125657Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-11-01T16:47:48.7142380Z Entering 'third_party/tensorpipe' 2024-11-01T16:47:48.7186001Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-11-01T16:47:48.7206681Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-11-01T16:47:48.7256481Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-11-01T16:47:48.7268946Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-11-01T16:47:48.7310946Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-11-01T16:47:48.7328732Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-11-01T16:47:48.7367611Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-11-01T16:47:48.7384272Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-11-01T16:47:48.7427297Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-11-01T16:47:48.7441084Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-11-01T16:47:48.7485116Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-11-01T16:47:48.8667412Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-11-01T16:47:48.8938025Z Entering 'android/libs/fbjni' 2024-11-01T16:47:48.8973999Z Entering 'third_party/FP16' 2024-11-01T16:47:48.9015738Z Entering 'third_party/FXdiv' 2024-11-01T16:47:48.9055331Z Entering 'third_party/NNPACK' 2024-11-01T16:47:48.9097184Z Entering 'third_party/NVTX' 2024-11-01T16:47:48.9140239Z Entering 'third_party/VulkanMemoryAllocator' 2024-11-01T16:47:48.9185628Z Entering 'third_party/XNNPACK' 2024-11-01T16:47:48.9240757Z Entering 'third_party/benchmark' 2024-11-01T16:47:48.9287532Z Entering 'third_party/composable_kernel' 2024-11-01T16:47:48.9336586Z Entering 'third_party/cpp-httplib' 2024-11-01T16:47:48.9385504Z Entering 'third_party/cpuinfo' 2024-11-01T16:47:48.9421403Z Entering 'third_party/cudnn_frontend' 2024-11-01T16:47:48.9467277Z Entering 'third_party/cutlass' 2024-11-01T16:47:48.9515631Z Entering 'third_party/eigen' 2024-11-01T16:47:48.9553260Z Entering 'third_party/fbgemm' 2024-11-01T16:47:48.9595847Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-11-01T16:47:48.9636954Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-11-01T16:47:48.9672015Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-11-01T16:47:48.9711592Z Entering 'third_party/fbgemm/third_party/googletest' 2024-11-01T16:47:48.9752912Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-11-01T16:47:48.9793567Z Entering 'third_party/flatbuffers' 2024-11-01T16:47:48.9839177Z Entering 'third_party/fmt' 2024-11-01T16:47:48.9880748Z Entering 'third_party/gemmlowp/gemmlowp' 2024-11-01T16:47:48.9923890Z Entering 'third_party/gloo' 2024-11-01T16:47:48.9962264Z Entering 'third_party/googletest' 2024-11-01T16:47:49.0002399Z Entering 'third_party/ideep' 2024-11-01T16:47:49.0043375Z Entering 'third_party/ideep/mkl-dnn' 2024-11-01T16:47:49.0091762Z Entering 'third_party/ittapi' 2024-11-01T16:47:49.0132526Z Entering 'third_party/kineto' 2024-11-01T16:47:49.0175130Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-11-01T16:47:49.0214711Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-11-01T16:47:49.0252349Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-11-01T16:47:49.0287549Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-11-01T16:47:49.0327349Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-11-01T16:47:49.0366343Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-11-01T16:47:49.0414000Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-11-01T16:47:49.0458737Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-11-01T16:47:49.0506629Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-11-01T16:47:49.0544229Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-11-01T16:47:49.0590433Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-11-01T16:47:49.0630970Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-11-01T16:47:49.0670773Z Entering 'third_party/mimalloc' 2024-11-01T16:47:49.0714785Z Entering 'third_party/nccl/nccl' 2024-11-01T16:47:49.0758116Z Entering 'third_party/nlohmann' 2024-11-01T16:47:49.0800396Z Entering 'third_party/onnx' 2024-11-01T16:47:49.0857928Z Entering 'third_party/onnx/third_party/pybind11' 2024-11-01T16:47:49.0902766Z Entering 'third_party/opentelemetry-cpp' 2024-11-01T16:47:49.0949223Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-11-01T16:47:49.0990196Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-11-01T16:47:49.1025886Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-11-01T16:47:49.1059744Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-11-01T16:47:49.1095869Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-11-01T16:47:49.1131886Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-11-01T16:47:49.1166061Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-11-01T16:47:49.1200562Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-11-01T16:47:49.1242778Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-11-01T16:47:49.1280449Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-11-01T16:47:49.1337894Z Entering 'third_party/pocketfft' 2024-11-01T16:47:49.1373404Z Entering 'third_party/protobuf' 2024-11-01T16:47:49.1428721Z Entering 'third_party/protobuf/third_party/benchmark' 2024-11-01T16:47:49.1463708Z Entering 'third_party/protobuf/third_party/googletest' 2024-11-01T16:47:49.1509667Z Entering 'third_party/psimd' 2024-11-01T16:47:49.1550477Z Entering 'third_party/pthreadpool' 2024-11-01T16:47:49.1590441Z Entering 'third_party/pybind11' 2024-11-01T16:47:49.1635422Z Entering 'third_party/python-peachpy' 2024-11-01T16:47:49.1680273Z Entering 'third_party/sleef' 2024-11-01T16:47:49.1720662Z Entering 'third_party/tensorpipe' 2024-11-01T16:47:49.1765016Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-11-01T16:47:49.1800067Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-11-01T16:47:49.1834747Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-11-01T16:47:49.1875934Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-11-01T16:47:49.1926464Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-11-01T16:47:49.1988146Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-11-01T16:47:49.2263391Z Entering 'android/libs/fbjni' 2024-11-01T16:47:49.2301325Z Entering 'third_party/FP16' 2024-11-01T16:47:49.2336257Z Entering 'third_party/FXdiv' 2024-11-01T16:47:49.2377645Z Entering 'third_party/NNPACK' 2024-11-01T16:47:49.2412348Z Entering 'third_party/NVTX' 2024-11-01T16:47:49.2449194Z Entering 'third_party/VulkanMemoryAllocator' 2024-11-01T16:47:49.2484366Z Entering 'third_party/XNNPACK' 2024-11-01T16:47:49.2536638Z Entering 'third_party/benchmark' 2024-11-01T16:47:49.2572479Z Entering 'third_party/composable_kernel' 2024-11-01T16:47:49.2611662Z Entering 'third_party/cpp-httplib' 2024-11-01T16:47:49.2647793Z Entering 'third_party/cpuinfo' 2024-11-01T16:47:49.2683169Z Entering 'third_party/cudnn_frontend' 2024-11-01T16:47:49.2718020Z Entering 'third_party/cutlass' 2024-11-01T16:47:49.2760829Z Entering 'third_party/eigen' 2024-11-01T16:47:49.2796086Z Entering 'third_party/fbgemm' 2024-11-01T16:47:49.2833760Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-11-01T16:47:49.2866151Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-11-01T16:47:49.2900684Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-11-01T16:47:49.2942895Z Entering 'third_party/fbgemm/third_party/googletest' 2024-11-01T16:47:49.2975799Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-11-01T16:47:49.3011627Z Entering 'third_party/flatbuffers' 2024-11-01T16:47:49.3048497Z Entering 'third_party/fmt' 2024-11-01T16:47:49.3083155Z Entering 'third_party/gemmlowp/gemmlowp' 2024-11-01T16:47:49.3117938Z Entering 'third_party/gloo' 2024-11-01T16:47:49.3154925Z Entering 'third_party/googletest' 2024-11-01T16:47:49.3190216Z Entering 'third_party/ideep' 2024-11-01T16:47:49.3223969Z Entering 'third_party/ideep/mkl-dnn' 2024-11-01T16:47:49.3273446Z Entering 'third_party/ittapi' 2024-11-01T16:47:49.3308031Z Entering 'third_party/kineto' 2024-11-01T16:47:49.3345729Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-11-01T16:47:49.3381452Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-11-01T16:47:49.3417189Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-11-01T16:47:49.3451988Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-11-01T16:47:49.3490359Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-11-01T16:47:49.3526271Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-11-01T16:47:49.3573881Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-11-01T16:47:49.3610318Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-11-01T16:47:49.3649300Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-11-01T16:47:49.3683878Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-11-01T16:47:49.3720337Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-11-01T16:47:49.3754229Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-11-01T16:47:49.3794721Z Entering 'third_party/mimalloc' 2024-11-01T16:47:49.3830864Z Entering 'third_party/nccl/nccl' 2024-11-01T16:47:49.3866588Z Entering 'third_party/nlohmann' 2024-11-01T16:47:49.3904231Z Entering 'third_party/onnx' 2024-11-01T16:47:49.3954533Z Entering 'third_party/onnx/third_party/pybind11' 2024-11-01T16:47:49.3999307Z Entering 'third_party/opentelemetry-cpp' 2024-11-01T16:47:49.4038317Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-11-01T16:47:49.4075949Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-11-01T16:47:49.4109492Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-11-01T16:47:49.4143504Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-11-01T16:47:49.4181053Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-11-01T16:47:49.4217274Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-11-01T16:47:49.4250250Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-11-01T16:47:49.4282291Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-11-01T16:47:49.4319887Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-11-01T16:47:49.4354732Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-11-01T16:47:49.4410360Z Entering 'third_party/pocketfft' 2024-11-01T16:47:49.4447587Z Entering 'third_party/protobuf' 2024-11-01T16:47:49.4484397Z Entering 'third_party/protobuf/third_party/benchmark' 2024-11-01T16:47:49.4520405Z Entering 'third_party/protobuf/third_party/googletest' 2024-11-01T16:47:49.4556648Z Entering 'third_party/psimd' 2024-11-01T16:47:49.4592501Z Entering 'third_party/pthreadpool' 2024-11-01T16:47:49.4629722Z Entering 'third_party/pybind11' 2024-11-01T16:47:49.4672932Z Entering 'third_party/python-peachpy' 2024-11-01T16:47:49.4706750Z Entering 'third_party/sleef' 2024-11-01T16:47:49.4744846Z Entering 'third_party/tensorpipe' 2024-11-01T16:47:49.4778648Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-11-01T16:47:49.4816379Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-11-01T16:47:49.4850944Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-11-01T16:47:49.4885430Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-11-01T16:47:49.4919214Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-11-01T16:47:49.4971854Z ##[endgroup] 2024-11-01T16:47:49.5004011Z [command]/usr/bin/git log -1 --format='%H' 2024-11-01T16:47:49.5030292Z 'd1aa4ef5a6c0ee2d57fb0086f5557e34537f0fea' 2024-11-01T16:47:49.5233422Z Prepare all required actions 2024-11-01T16:47:49.5233953Z Getting action download info 2024-11-01T16:47:49.6839512Z ##[group]Run ./.github/actions/setup-linux 2024-11-01T16:47:49.6839983Z env: 2024-11-01T16:47:49.6840297Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:49.6840684Z ##[endgroup] 2024-11-01T16:47:49.6909769Z ##[group]Run set -euo pipefail 2024-11-01T16:47:49.6910452Z set -euo pipefail 2024-11-01T16:47:49.6910887Z function get_ec2_metadata() { 2024-11-01T16:47:49.6911514Z  # Pulled from instance metadata endpoint for EC2 2024-11-01T16:47:49.6912584Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-11-01T16:47:49.6913509Z  category=$1 2024-11-01T16:47:49.6914126Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-11-01T16:47:49.6914896Z  runner_name_str=i-0fb054b0c783c31fa 2024-11-01T16:47:49.6915490Z  if [[ -f /.inarc ]]; then 2024-11-01T16:47:49.6916078Z  echo "ARC Runner, no info on ec2 metadata" 2024-11-01T16:47:49.6916721Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-11-01T16:47:49.6917545Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-11-01T16:47:49.6918275Z  else 2024-11-01T16:47:49.6919746Z  curl -H "X-aws-ec2-metadata-token: $(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 30")" -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-11-01T16:47:49.6921295Z  fi 2024-11-01T16:47:49.6921737Z } 2024-11-01T16:47:49.6922259Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-11-01T16:47:49.6922868Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-11-01T16:47:49.6923584Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-11-01T16:47:49.6924206Z echo "system info $(uname -a)" 2024-11-01T16:47:49.6930885Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:49.6931462Z env: 2024-11-01T16:47:49.6931776Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:49.6932131Z ##[endgroup] 2024-11-01T16:47:49.7091904Z ami-id: ami-0fff1b9a61dec8a5f 2024-11-01T16:47:49.7185202Z instance-id: i-0fb054b0c783c31fa 2024-11-01T16:47:49.7282205Z instance-type: g4dn.metal 2024-11-01T16:47:49.7291985Z system info Linux ip-10-0-54-33.ec2.internal 6.1.109-118.189.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Sep 10 08:59:12 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-11-01T16:47:49.7319949Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2024-11-01T16:47:49.7321334Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2024-11-01T16:47:49.7327294Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:49.7327811Z env: 2024-11-01T16:47:49.7328079Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:49.7328423Z ##[endgroup] 2024-11-01T16:47:49.7381753Z ##[group]Run if systemctl is-active --quiet docker; then 2024-11-01T16:47:49.7382403Z if systemctl is-active --quiet docker; then 2024-11-01T16:47:49.7382993Z  echo "Docker daemon is running..."; 2024-11-01T16:47:49.7383479Z else 2024-11-01T16:47:49.7384009Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-11-01T16:47:49.7384647Z fi 2024-11-01T16:47:49.7389307Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:49.7389839Z env: 2024-11-01T16:47:49.7390125Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:49.7390491Z ##[endgroup] 2024-11-01T16:47:49.7465159Z Docker daemon is running... 2024-11-01T16:47:49.7514297Z ##[group]Run nick-fields/retry@v3.0.0 2024-11-01T16:47:49.7514737Z with: 2024-11-01T16:47:49.7515020Z shell: bash 2024-11-01T16:47:49.7515490Z timeout_minutes: 5 2024-11-01T16:47:49.7515843Z max_attempts: 3 2024-11-01T16:47:49.7516182Z retry_wait_seconds: 30 2024-11-01T16:47:49.7519817Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-11-01T16:47:49.7523610Z polling_interval_seconds: 1 2024-11-01T16:47:49.7523989Z warning_on_retry: true 2024-11-01T16:47:49.7524362Z continue_on_error: false 2024-11-01T16:47:49.7524698Z env: 2024-11-01T16:47:49.7524954Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:49.7525309Z AWS_RETRY_MODE: standard 2024-11-01T16:47:49.7525656Z AWS_MAX_ATTEMPTS: 5 2024-11-01T16:47:49.7525994Z AWS_DEFAULT_REGION: us-east-1 2024-11-01T16:47:49.7526368Z ##[endgroup] 2024-11-01T16:47:50.9690930Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-11-01T16:47:50.9693347Z Configure a credential helper to remove this warning. See 2024-11-01T16:47:50.9694325Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-11-01T16:47:50.9694926Z 2024-11-01T16:47:50.9695083Z Login Succeeded 2024-11-01T16:47:51.9121173Z Command completed after 1 attempt(s). 2024-11-01T16:47:51.9203911Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-11-01T16:47:51.9204760Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-11-01T16:47:51.9205559Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-11-01T16:47:51.9212694Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:51.9213281Z env: 2024-11-01T16:47:51.9213605Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:51.9214015Z ##[endgroup] 2024-11-01T16:47:51.9302745Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-11-01T16:47:51.9303785Z # ignore expansion of "docker ps -q" since it could be empty 2024-11-01T16:47:51.9304470Z # shellcheck disable=SC2046 2024-11-01T16:47:51.9305003Z docker stop $(docker ps -q) || true 2024-11-01T16:47:51.9305540Z # Prune all of the docker images 2024-11-01T16:47:51.9306049Z docker system prune -af 2024-11-01T16:47:51.9312414Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:51.9312982Z env: 2024-11-01T16:47:51.9313299Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:51.9313690Z ##[endgroup] 2024-11-01T16:47:51.9593245Z "docker stop" requires at least 1 argument. 2024-11-01T16:47:51.9594084Z See 'docker stop --help'. 2024-11-01T16:47:51.9594368Z 2024-11-01T16:47:51.9594641Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-11-01T16:47:51.9595098Z 2024-11-01T16:47:51.9595271Z Stop one or more running containers 2024-11-01T16:47:51.9755130Z Total reclaimed space: 0B 2024-11-01T16:47:51.9792542Z ##[group]Run set +e 2024-11-01T16:47:51.9792939Z set +e 2024-11-01T16:47:51.9793261Z set -x 2024-11-01T16:47:51.9793575Z  2024-11-01T16:47:51.9793934Z PT_DOMAIN=download.pytorch.org 2024-11-01T16:47:51.9794845Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-11-01T16:47:51.9796135Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-11-01T16:47:51.9797019Z # one is returned at random 2024-11-01T16:47:51.9797641Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-11-01T16:47:51.9798352Z  2024-11-01T16:47:51.9798842Z if [ -z "${RESOLVED_IP}" ]; then 2024-11-01T16:47:51.9799542Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-11-01T16:47:51.9800409Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-11-01T16:47:51.9801034Z  2024-11-01T16:47:51.9801362Z  if [ -z "${RESOLVED_IP}" ]; then 2024-11-01T16:47:51.9801971Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-11-01T16:47:51.9802545Z  exit 1 2024-11-01T16:47:51.9802886Z  fi 2024-11-01T16:47:51.9803202Z fi 2024-11-01T16:47:51.9803493Z  2024-11-01T16:47:51.9803868Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-11-01T16:47:51.9804441Z  # Clean up any old records first 2024-11-01T16:47:51.9804999Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-11-01T16:47:51.9805498Z fi 2024-11-01T16:47:51.9805786Z  2024-11-01T16:47:51.9806243Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-11-01T16:47:51.9806855Z cat /etc/hosts 2024-11-01T16:47:51.9812787Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:51.9813331Z env: 2024-11-01T16:47:51.9813627Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:51.9814002Z ##[endgroup] 2024-11-01T16:47:51.9836550Z + PT_DOMAIN=download.pytorch.org 2024-11-01T16:47:51.9845618Z ++ dig -4 +short download.pytorch.org 2024-11-01T16:47:51.9846136Z ++ tail -n1 2024-11-01T16:47:52.0098649Z + RESOLVED_IP=18.160.10.28 2024-11-01T16:47:52.0099242Z + '[' -z 18.160.10.28 ']' 2024-11-01T16:47:52.0099741Z + grep -r download.pytorch.org /etc/hosts 2024-11-01T16:47:52.0109440Z 18.160.10.36 download.pytorch.org 2024-11-01T16:47:52.0111740Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-11-01T16:47:52.1081110Z + echo '18.160.10.28 download.pytorch.org' 2024-11-01T16:47:52.1081702Z + sudo tee -a /etc/hosts 2024-11-01T16:47:52.1296820Z 18.160.10.28 download.pytorch.org 2024-11-01T16:47:52.1311861Z + cat /etc/hosts 2024-11-01T16:47:52.1317811Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-11-01T16:47:52.1329709Z ::1 localhost6 localhost6.localdomain6 2024-11-01T16:47:52.1330351Z 18.160.10.28 download.pytorch.org 2024-11-01T16:47:52.1532602Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-11-01T16:47:52.1533229Z with: 2024-11-01T16:47:52.1534448Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.1535809Z docker-build-dir: .ci/docker 2024-11-01T16:47:52.1536188Z working-directory: . 2024-11-01T16:47:52.1536664Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.1537202Z force-push: false 2024-11-01T16:47:52.1537503Z env: 2024-11-01T16:47:52.1537768Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:52.1538104Z ##[endgroup] 2024-11-01T16:47:52.1560334Z ##[group]Run set -ex 2024-11-01T16:47:52.1560736Z set -ex 2024-11-01T16:47:52.1561052Z  2024-11-01T16:47:52.1561655Z # If the docker build directory or the build script doesn't exist, the action will 2024-11-01T16:47:52.1562773Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-11-01T16:47:52.1563647Z # job could then download the pre-built image as usual 2024-11-01T16:47:52.1564436Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-11-01T16:47:52.1565171Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-11-01T16:47:52.1565857Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-11-01T16:47:52.1566463Z  2024-11-01T16:47:52.1567015Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-11-01T16:47:52.1567817Z  exit 0 2024-11-01T16:47:52.1568120Z else 2024-11-01T16:47:52.1568510Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-11-01T16:47:52.1568993Z fi 2024-11-01T16:47:52.1569291Z  2024-11-01T16:47:52.1569802Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-11-01T16:47:52.1570728Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-11-01T16:47:52.1571553Z  # use it as it is, but first let's extract the tag 2024-11-01T16:47:52.1572317Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-11-01T16:47:52.1573106Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-11-01T16:47:52.1573867Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-11-01T16:47:52.1574478Z else 2024-11-01T16:47:52.1574939Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-11-01T16:47:52.1575654Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-11-01T16:47:52.1576606Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-11-01T16:47:52.1601405Z fi 2024-11-01T16:47:52.1607325Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:52.1607884Z env: 2024-11-01T16:47:52.1608187Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:52.1608561Z REPO_NAME: pytorch 2024-11-01T16:47:52.1609965Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.1611608Z DOCKER_BUILD_DIR: .ci/docker 2024-11-01T16:47:52.1612180Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.1612776Z ##[endgroup] 2024-11-01T16:47:52.1638730Z + [[ ! -d .ci/docker ]] 2024-11-01T16:47:52.1640012Z + [[ ! -f .ci/docker/build.sh ]] 2024-11-01T16:47:52.1640881Z + echo skip=false 2024-11-01T16:47:52.1643664Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-11-01T16:47:52.1647083Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.1648539Z ++ awk -F '[:,]' '{print $2}' 2024-11-01T16:47:52.1667167Z + DOCKER_TAG=bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.1668036Z + echo docker-tag=bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.1670018Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.1755747Z ##[group]Run set +e 2024-11-01T16:47:52.1756133Z set +e 2024-11-01T16:47:52.1756450Z set -x 2024-11-01T16:47:52.1756757Z  2024-11-01T16:47:52.1757052Z login() { 2024-11-01T16:47:52.1757777Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-11-01T16:47:52.1758538Z } 2024-11-01T16:47:52.1758834Z  2024-11-01T16:47:52.1759126Z retry () { 2024-11-01T16:47:52.1759537Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-11-01T16:47:52.1760005Z } 2024-11-01T16:47:52.1760294Z  2024-11-01T16:47:52.1760605Z retry login "${DOCKER_REGISTRY}" 2024-11-01T16:47:52.1761046Z  2024-11-01T16:47:52.1761553Z # Check if image already exists, if it does then skip building it 2024-11-01T16:47:52.1762335Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-11-01T16:47:52.1763054Z  exit 0 2024-11-01T16:47:52.1763359Z fi 2024-11-01T16:47:52.1763663Z  2024-11-01T16:47:52.1764216Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-11-01T16:47:52.1765132Z # be empty. The default action would be to continue rebuild the image 2024-11-01T16:47:52.1765944Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-11-01T16:47:52.1766674Z  # if we're on the base branch then use the parent commit 2024-11-01T16:47:52.1767293Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-11-01T16:47:52.1767761Z else 2024-11-01T16:47:52.1768274Z  # otherwise we're on a PR, so use the most recent base commit 2024-11-01T16:47:52.1769030Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-11-01T16:47:52.1769582Z fi 2024-11-01T16:47:52.1769884Z  2024-11-01T16:47:52.1770208Z if [[ -z "${MERGE_BASE}" ]]; then 2024-11-01T16:47:52.1770769Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-11-01T16:47:52.1771269Z  2024-11-01T16:47:52.1771983Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-11-01T16:47:52.1772829Z  exit 0 2024-11-01T16:47:52.1773161Z fi 2024-11-01T16:47:52.1773435Z  2024-11-01T16:47:52.1773920Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-11-01T16:47:52.1774991Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-11-01T16:47:52.1775887Z  exit 1 2024-11-01T16:47:52.1776216Z fi 2024-11-01T16:47:52.1776493Z  2024-11-01T16:47:52.1777031Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-11-01T16:47:52.1778083Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-11-01T16:47:52.1779047Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-11-01T16:47:52.1780109Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-11-01T16:47:52.1781314Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-11-01T16:47:52.1782143Z fi 2024-11-01T16:47:52.1782427Z  2024-11-01T16:47:52.1782811Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-11-01T16:47:52.1787993Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:52.1788537Z env: 2024-11-01T16:47:52.1788837Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:52.1789235Z DOCKER_BUILD_DIR: .ci/docker 2024-11-01T16:47:52.1789705Z BASE_REVISION: 2055d3c8dc8b93c52eda6b9c07d716448d9e28c9 2024-11-01T16:47:52.1791146Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.1792569Z DOCKER_TAG: bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.1793217Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.1793781Z ##[endgroup] 2024-11-01T16:47:52.1817407Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.1818224Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.1819258Z + aws ecr get-login-password --region us-east-1 2024-11-01T16:47:52.1820213Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.6933216Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-11-01T16:47:52.6934176Z Configure a credential helper to remove this warning. See 2024-11-01T16:47:52.6935143Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-11-01T16:47:52.6935762Z 2024-11-01T16:47:52.6935882Z Login Succeeded 2024-11-01T16:47:52.6959811Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.9253167Z { 2024-11-01T16:47:52.9253873Z "schemaVersion": 2, 2024-11-01T16:47:52.9254534Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-11-01T16:47:52.9255222Z "config": { 2024-11-01T16:47:52.9255710Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-11-01T16:47:52.9256334Z "size": 50538, 2024-11-01T16:47:52.9256954Z "digest": "sha256:235d628183ab1dfb4496bc63787f13b256e54f8c1879b1682130b9b8021e973b" 2024-11-01T16:47:52.9257654Z }, 2024-11-01T16:47:52.9257925Z "layers": [ 2024-11-01T16:47:52.9258209Z { 2024-11-01T16:47:52.9258677Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9259318Z "size": 28580681, 2024-11-01T16:47:52.9259961Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-11-01T16:47:52.9260804Z }, 2024-11-01T16:47:52.9261057Z { 2024-11-01T16:47:52.9261805Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9262439Z "size": 7943451, 2024-11-01T16:47:52.9263053Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-11-01T16:47:52.9263783Z }, 2024-11-01T16:47:52.9264041Z { 2024-11-01T16:47:52.9264550Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9265179Z "size": 55728572, 2024-11-01T16:47:52.9265818Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-11-01T16:47:52.9266518Z }, 2024-11-01T16:47:52.9266773Z { 2024-11-01T16:47:52.9267252Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9267875Z "size": 186, 2024-11-01T16:47:52.9268497Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-11-01T16:47:52.9269241Z }, 2024-11-01T16:47:52.9269499Z { 2024-11-01T16:47:52.9269977Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9270800Z "size": 6886, 2024-11-01T16:47:52.9271450Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-11-01T16:47:52.9272179Z }, 2024-11-01T16:47:52.9272655Z { 2024-11-01T16:47:52.9273153Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9273809Z "size": 1329001756, 2024-11-01T16:47:52.9274465Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-11-01T16:47:52.9275206Z }, 2024-11-01T16:47:52.9275465Z { 2024-11-01T16:47:52.9275945Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9276582Z "size": 62501, 2024-11-01T16:47:52.9277186Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-11-01T16:47:52.9278328Z + exit 0 2024-11-01T16:47:52.9278608Z }, 2024-11-01T16:47:52.9278862Z { 2024-11-01T16:47:52.9279340Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9280109Z "size": 1684, 2024-11-01T16:47:52.9280734Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-11-01T16:47:52.9281426Z }, 2024-11-01T16:47:52.9281694Z { 2024-11-01T16:47:52.9282173Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9282803Z "size": 1523, 2024-11-01T16:47:52.9283440Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-11-01T16:47:52.9284163Z }, 2024-11-01T16:47:52.9284422Z { 2024-11-01T16:47:52.9284898Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9285528Z "size": 2528295403, 2024-11-01T16:47:52.9286175Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-11-01T16:47:52.9288774Z }, 2024-11-01T16:47:52.9289034Z { 2024-11-01T16:47:52.9289518Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9290149Z "size": 86016, 2024-11-01T16:47:52.9290787Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-11-01T16:47:52.9291620Z }, 2024-11-01T16:47:52.9291854Z { 2024-11-01T16:47:52.9292322Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9292929Z "size": 1823, 2024-11-01T16:47:52.9293542Z "digest": "sha256:671dad96a4d7d88ce4bed021433e38ec73e84fa9fa4b65c0a18fbcc457591ea6" 2024-11-01T16:47:52.9294245Z }, 2024-11-01T16:47:52.9294482Z { 2024-11-01T16:47:52.9294951Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9295567Z "size": 247040648, 2024-11-01T16:47:52.9296200Z "digest": "sha256:ea0146d98a6522d2c356bac97d2ccab07d3fff9c64a23ba1e909e0190926950c" 2024-11-01T16:47:52.9296906Z }, 2024-11-01T16:47:52.9297148Z { 2024-11-01T16:47:52.9297616Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9298227Z "size": 704, 2024-11-01T16:47:52.9298798Z "digest": "sha256:68392a17394664e4dda99f6336695a5199211cd55de7c435d783c2907f93813d" 2024-11-01T16:47:52.9299470Z }, 2024-11-01T16:47:52.9299707Z { 2024-11-01T16:47:52.9300184Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9300802Z "size": 1257, 2024-11-01T16:47:52.9301398Z "digest": "sha256:da40e3f2b2b4f69e8104646105a28f7002dc35e7cf05236e6a4214f5b92d2a09" 2024-11-01T16:47:52.9302084Z }, 2024-11-01T16:47:52.9302322Z { 2024-11-01T16:47:52.9302794Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9303406Z "size": 484, 2024-11-01T16:47:52.9303982Z "digest": "sha256:0794764317b2d8b884fa3461280817ec3fb1448e5f69cb37654f3c8a75a5a25d" 2024-11-01T16:47:52.9304651Z }, 2024-11-01T16:47:52.9304901Z { 2024-11-01T16:47:52.9305365Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9305991Z "size": 91712833, 2024-11-01T16:47:52.9306607Z "digest": "sha256:9652f7171409f2578e30e7bf4cbdbae6d6c2a66bd51c1edb0696cb791ab1f201" 2024-11-01T16:47:52.9307301Z }, 2024-11-01T16:47:52.9307553Z { 2024-11-01T16:47:52.9308005Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9308703Z "size": 3678, 2024-11-01T16:47:52.9309305Z "digest": "sha256:a65906210ebf495dabd48d68dc6312650d47250bf678ab080e1d55307091cf25" 2024-11-01T16:47:52.9310373Z }, 2024-11-01T16:47:52.9310636Z { 2024-11-01T16:47:52.9311105Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9311743Z "size": 1801, 2024-11-01T16:47:52.9312366Z "digest": "sha256:216dc81274cf0d8a0bd17caeb0f04287df4dfee19b7e39191671835a252fdb30" 2024-11-01T16:47:52.9313089Z }, 2024-11-01T16:47:52.9313346Z { 2024-11-01T16:47:52.9313814Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9314453Z "size": 702, 2024-11-01T16:47:52.9315075Z "digest": "sha256:68ddca4853417c8bc2ab42ebf9e028c1ef981bcb8c23cda38687000e5607bcd0" 2024-11-01T16:47:52.9315790Z }, 2024-11-01T16:47:52.9316043Z { 2024-11-01T16:47:52.9316522Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9317141Z "size": 2893545452, 2024-11-01T16:47:52.9317783Z "digest": "sha256:cc6151119a1f6176fd9683c0e9a85c1c8e3b63e759a8baf6b6ce12994f902770" 2024-11-01T16:47:52.9318492Z }, 2024-11-01T16:47:52.9318750Z { 2024-11-01T16:47:52.9319229Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9319845Z "size": 380, 2024-11-01T16:47:52.9320453Z "digest": "sha256:3f19dbe230a99e91a19bd81e76adf3227aa08b48791471153602f3ea86969ebf" 2024-11-01T16:47:52.9321161Z }, 2024-11-01T16:47:52.9321418Z { 2024-11-01T16:47:52.9322008Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9322709Z "size": 12863, 2024-11-01T16:47:52.9323307Z "digest": "sha256:29115f672c890d759a7c5577c2b9fb4c6b91f4319626e58f703efa128e20f3b9" 2024-11-01T16:47:52.9323988Z }, 2024-11-01T16:47:52.9324240Z { 2024-11-01T16:47:52.9324712Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9325325Z "size": 863, 2024-11-01T16:47:52.9325931Z "digest": "sha256:d713d505e57eed91b0942e5bceb5833b64cd001db16797cb018c750362919e0f" 2024-11-01T16:47:52.9326622Z }, 2024-11-01T16:47:52.9326870Z { 2024-11-01T16:47:52.9327333Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9327946Z "size": 106, 2024-11-01T16:47:52.9328526Z "digest": "sha256:668e2ec2d1a6c61998850a9cc0d447390a02bbf7cd5a06c63c54c20ef6cec099" 2024-11-01T16:47:52.9329214Z }, 2024-11-01T16:47:52.9329463Z { 2024-11-01T16:47:52.9329929Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9330537Z "size": 504, 2024-11-01T16:47:52.9331114Z "digest": "sha256:b30b6fc0415d34c77725c97a9150d7aa1962b3d8b9092fe722356d947ce045e2" 2024-11-01T16:47:52.9331798Z }, 2024-11-01T16:47:52.9332047Z { 2024-11-01T16:47:52.9332513Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9333129Z "size": 121477286, 2024-11-01T16:47:52.9333740Z "digest": "sha256:e3f18d02cbdf1344566f178f6a430c810c44f6ad334e3b24be47d82cb96d5022" 2024-11-01T16:47:52.9334433Z }, 2024-11-01T16:47:52.9334682Z { 2024-11-01T16:47:52.9335148Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9335760Z "size": 109, 2024-11-01T16:47:52.9336337Z "digest": "sha256:399a6aa7d4a7edc6567b3c805b537367a752050b86bd87eb169b9579ece9009f" 2024-11-01T16:47:52.9337025Z }, 2024-11-01T16:47:52.9337274Z { 2024-11-01T16:47:52.9337736Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9338352Z "size": 490, 2024-11-01T16:47:52.9338941Z "digest": "sha256:7f31c9564805381ee698a8f46a33f68eddf129117ce0ae67fe0d967a6a0d9507" 2024-11-01T16:47:52.9339615Z }, 2024-11-01T16:47:52.9339867Z { 2024-11-01T16:47:52.9340332Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9340939Z "size": 297, 2024-11-01T16:47:52.9341530Z "digest": "sha256:58666e413ec3c9e8e81cb369d2d03425f95020df6d3c5ff10ac72e05b02610e8" 2024-11-01T16:47:52.9342307Z }, 2024-11-01T16:47:52.9342567Z { 2024-11-01T16:47:52.9343070Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9343725Z "size": 103, 2024-11-01T16:47:52.9344354Z "digest": "sha256:4f4eab9b0a12f66822619740d353a19c56c73945d1399be55f68cef87d6fe33c" 2024-11-01T16:47:52.9345044Z }, 2024-11-01T16:47:52.9345339Z { 2024-11-01T16:47:52.9345839Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9346480Z "size": 1474, 2024-11-01T16:47:52.9347103Z "digest": "sha256:435d64b6b0044737aca81336317bc4c9ae901a357051cd3ad1a0255de29b8e71" 2024-11-01T16:47:52.9347801Z }, 2024-11-01T16:47:52.9348081Z { 2024-11-01T16:47:52.9348577Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9349222Z "size": 426740291, 2024-11-01T16:47:52.9349903Z "digest": "sha256:1dc2a0d3d3bad5cbaa34bdccce897ad82abbc2fb970130839e27c8a2327800ec" 2024-11-01T16:47:52.9350656Z }, 2024-11-01T16:47:52.9350919Z { 2024-11-01T16:47:52.9351411Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9352057Z "size": 164, 2024-11-01T16:47:52.9352690Z "digest": "sha256:6d4c8df84ceeb67735a6215ae68aab6ce5a1152d3f06b458f6a5a4c2a23c45a0" 2024-11-01T16:47:52.9353427Z }, 2024-11-01T16:47:52.9353684Z { 2024-11-01T16:47:52.9354182Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9354826Z "size": 565, 2024-11-01T16:47:52.9355453Z "digest": "sha256:b65cd1b6276b6a9458dacb21ec97518828783bfba467697b14c29d10b62b9c07" 2024-11-01T16:47:52.9356253Z }, 2024-11-01T16:47:52.9356517Z { 2024-11-01T16:47:52.9357018Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9357673Z "size": 35874313, 2024-11-01T16:47:52.9358342Z "digest": "sha256:ee2520d1a68e8d6dcdc364f6ca63c8eacd9410adbb4e453bc668696bc8373c7d" 2024-11-01T16:47:52.9359089Z }, 2024-11-01T16:47:52.9359349Z { 2024-11-01T16:47:52.9359852Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9360504Z "size": 104, 2024-11-01T16:47:52.9361142Z "digest": "sha256:e9a23ac9bc630078b40a26f7edc1ff92bb3d8ddcbde1b1044db6904818045bcd" 2024-11-01T16:47:52.9361880Z }, 2024-11-01T16:47:52.9362129Z { 2024-11-01T16:47:52.9362624Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9363265Z "size": 426, 2024-11-01T16:47:52.9363897Z "digest": "sha256:5bf4f6bf261cfb578868c1baf62db3571dd5cff7028365f49c6e3f258a7c11bb" 2024-11-01T16:47:52.9364622Z }, 2024-11-01T16:47:52.9364904Z { 2024-11-01T16:47:52.9365380Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9366025Z "size": 20262177, 2024-11-01T16:47:52.9366657Z "digest": "sha256:1c82010b28c5dc5f21d8182d28d438db058112e70d92919557187cee26466e7f" 2024-11-01T16:47:52.9367368Z }, 2024-11-01T16:47:52.9367651Z { 2024-11-01T16:47:52.9368124Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9368764Z "size": 440, 2024-11-01T16:47:52.9369375Z "digest": "sha256:6d6440d71b36004b9ed94d629d53d527a2c32c497ee6d73b826f19b76f10f8f8" 2024-11-01T16:47:52.9370081Z }, 2024-11-01T16:47:52.9370354Z { 2024-11-01T16:47:52.9370823Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9371461Z "size": 702, 2024-11-01T16:47:52.9372086Z "digest": "sha256:68ddca4853417c8bc2ab42ebf9e028c1ef981bcb8c23cda38687000e5607bcd0" 2024-11-01T16:47:52.9372812Z }, 2024-11-01T16:47:52.9373086Z { 2024-11-01T16:47:52.9373558Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9374208Z "size": 141, 2024-11-01T16:47:52.9374838Z "digest": "sha256:4ef0ae975192cfbaae71bb5a6081a8608cf98ff324335e023828dd1da3184630" 2024-11-01T16:47:52.9375564Z }, 2024-11-01T16:47:52.9375843Z { 2024-11-01T16:47:52.9376345Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9377060Z "size": 135, 2024-11-01T16:47:52.9377695Z "digest": "sha256:783687f05ffaad17a790ff0fca24e800b840de1b9c50d3b96e92362b03415bf8" 2024-11-01T16:47:52.9378419Z }, 2024-11-01T16:47:52.9378709Z { 2024-11-01T16:47:52.9379211Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9379834Z "size": 5036197630, 2024-11-01T16:47:52.9380645Z "digest": "sha256:a76f47476cd0a5635adcf444f8a2e27aef9be42764bb5accd0ccaadd5617a4dc" 2024-11-01T16:47:52.9381372Z }, 2024-11-01T16:47:52.9381651Z { 2024-11-01T16:47:52.9382139Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9382765Z "size": 195, 2024-11-01T16:47:52.9383383Z "digest": "sha256:71aac5529f94f92db0485ee5e69fe50958300d635b7b31c4abfc72d0b1284d68" 2024-11-01T16:47:52.9384077Z }, 2024-11-01T16:47:52.9384323Z { 2024-11-01T16:47:52.9384789Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9385389Z "size": 566, 2024-11-01T16:47:52.9385999Z "digest": "sha256:fa02b3beb343c16519ede66739898698825ca4c251a5424ac23068bb65639949" 2024-11-01T16:47:52.9386713Z }, 2024-11-01T16:47:52.9386985Z { 2024-11-01T16:47:52.9387478Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9388117Z "size": 43163277, 2024-11-01T16:47:52.9388734Z "digest": "sha256:51910ec2cd8fbcf6b79564371697d6fc3f0ca20765dc5b205f05a3574b11a6fc" 2024-11-01T16:47:52.9389443Z }, 2024-11-01T16:47:52.9389691Z { 2024-11-01T16:47:52.9390153Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9390829Z "size": 106, 2024-11-01T16:47:52.9391439Z "digest": "sha256:0fac4ecd0f3ee6c267bbfe219dfb37dab8a8130066d5cec8312fd32b3f69a419" 2024-11-01T16:47:52.9392159Z }, 2024-11-01T16:47:52.9392410Z { 2024-11-01T16:47:52.9392879Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9393512Z "size": 1403, 2024-11-01T16:47:52.9394098Z "digest": "sha256:402d8839fdba0a7827a6a059471bbae6088e07c3538de8017f93b32affa9950b" 2024-11-01T16:47:52.9394784Z }, 2024-11-01T16:47:52.9395034Z { 2024-11-01T16:47:52.9395497Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9396147Z "size": 702, 2024-11-01T16:47:52.9396753Z "digest": "sha256:68ddca4853417c8bc2ab42ebf9e028c1ef981bcb8c23cda38687000e5607bcd0" 2024-11-01T16:47:52.9397450Z }, 2024-11-01T16:47:52.9397709Z { 2024-11-01T16:47:52.9398172Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9398791Z "size": 136, 2024-11-01T16:47:52.9399389Z "digest": "sha256:af8723b4a86ff2ea6c385964fbcf1609516c9548eb620f244667e32216c26c0a" 2024-11-01T16:47:52.9400082Z }, 2024-11-01T16:47:52.9400328Z { 2024-11-01T16:47:52.9400792Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9401408Z "size": 120, 2024-11-01T16:47:52.9402015Z "digest": "sha256:7389530cf6bda265f2dac3fca59878096bb2699ae9e1a1438b99eb4689b26c6f" 2024-11-01T16:47:52.9402692Z }, 2024-11-01T16:47:52.9402952Z { 2024-11-01T16:47:52.9403419Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9404033Z "size": 4299151374, 2024-11-01T16:47:52.9404668Z "digest": "sha256:e138b7611c1cdc58776b8c30edbcfaab3a4a732f1c0aac549460fad646bf3a6c" 2024-11-01T16:47:52.9405350Z }, 2024-11-01T16:47:52.9405599Z { 2024-11-01T16:47:52.9406061Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9406667Z "size": 176, 2024-11-01T16:47:52.9407248Z "digest": "sha256:7449ddbd03b486c1f01960300ce162038b98031e870e497cfafddcfc7454035e" 2024-11-01T16:47:52.9407926Z }, 2024-11-01T16:47:52.9408181Z { 2024-11-01T16:47:52.9408646Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9409257Z "size": 907, 2024-11-01T16:47:52.9410087Z "digest": "sha256:dc59e6876e819427d034efe87187fef0c130c07d98eef1438c6f9be801ebaae5" 2024-11-01T16:47:52.9410912Z }, 2024-11-01T16:47:52.9411158Z { 2024-11-01T16:47:52.9411642Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9412272Z "size": 702, 2024-11-01T16:47:52.9412911Z "digest": "sha256:68ddca4853417c8bc2ab42ebf9e028c1ef981bcb8c23cda38687000e5607bcd0" 2024-11-01T16:47:52.9413637Z }, 2024-11-01T16:47:52.9413895Z { 2024-11-01T16:47:52.9414384Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9415008Z "size": 135, 2024-11-01T16:47:52.9415628Z "digest": "sha256:3ccf55ddb42851668f8b75c45d363e9b0f1fcccbbb16ed44c5ea2ff5fe130251" 2024-11-01T16:47:52.9416351Z }, 2024-11-01T16:47:52.9416591Z { 2024-11-01T16:47:52.9417073Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9417711Z "size": 32, 2024-11-01T16:47:52.9418319Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-11-01T16:47:52.9419030Z }, 2024-11-01T16:47:52.9419285Z { 2024-11-01T16:47:52.9419785Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9420428Z "size": 158, 2024-11-01T16:47:52.9421035Z "digest": "sha256:7367c224890d0427d5bf259b8b93d8cb1a0b12b09396c54606277cef516f8f6e" 2024-11-01T16:47:52.9421744Z }, 2024-11-01T16:47:52.9422242Z { 2024-11-01T16:47:52.9422677Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9423268Z "size": 2047, 2024-11-01T16:47:52.9423824Z "digest": "sha256:9ea471a9650bfc18020c810e31438225ea677629063c8d34c1d1485c284d415a" 2024-11-01T16:47:52.9424456Z }, 2024-11-01T16:47:52.9424759Z { 2024-11-01T16:47:52.9425188Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9425770Z "size": 7529783, 2024-11-01T16:47:52.9426341Z "digest": "sha256:5ac6db3b2d603f7494ad8e3329373e40f01620ee7a2d0161cdd558846c31b892" 2024-11-01T16:47:52.9426996Z }, 2024-11-01T16:47:52.9427239Z { 2024-11-01T16:47:52.9427678Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9428278Z "size": 164, 2024-11-01T16:47:52.9428829Z "digest": "sha256:0eb949a277118d4b8f7114fd37ecdc981dd79cb7117d8923e349182ae515882e" 2024-11-01T16:47:52.9429482Z }, 2024-11-01T16:47:52.9429713Z { 2024-11-01T16:47:52.9430136Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9430710Z "size": 7944, 2024-11-01T16:47:52.9431284Z "digest": "sha256:b9ba726a08563e3afcd2ad4c83963f4bedf2004d508dd676fec41662ae43b53a" 2024-11-01T16:47:52.9431939Z }, 2024-11-01T16:47:52.9432172Z { 2024-11-01T16:47:52.9432610Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9433167Z "size": 8070, 2024-11-01T16:47:52.9433731Z "digest": "sha256:9760655c3b12ec98e8067dfe3642290b2df828f2eb499abb2ca615348638bfb4" 2024-11-01T16:47:52.9434370Z }, 2024-11-01T16:47:52.9434602Z { 2024-11-01T16:47:52.9435048Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9435608Z "size": 303, 2024-11-01T16:47:52.9436168Z "digest": "sha256:62a8fbb6691e35aeabd37666cb694a2e6d61682e8f8e2dbb2892b09f2c78f788" 2024-11-01T16:47:52.9436823Z }, 2024-11-01T16:47:52.9437064Z { 2024-11-01T16:47:52.9437511Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9438082Z "size": 7630047, 2024-11-01T16:47:52.9438654Z "digest": "sha256:e7e230f57c57f45ad6655b4c165fa5e31cb5dac0b433d3f605b9d51990eae46d" 2024-11-01T16:47:52.9439304Z }, 2024-11-01T16:47:52.9439548Z { 2024-11-01T16:47:52.9440005Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9440583Z "size": 108, 2024-11-01T16:47:52.9441153Z "digest": "sha256:10e81741e898bd9dccd97375be804c88cc352919db4f3d522749c4b4f2b757c7" 2024-11-01T16:47:52.9441800Z }, 2024-11-01T16:47:52.9442037Z { 2024-11-01T16:47:52.9442495Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9443162Z "size": 54145661, 2024-11-01T16:47:52.9443762Z "digest": "sha256:b1aaa169430e16a55ce6cb2c963f113acee60b0359ef473c90f02aa94e39dfd8" 2024-11-01T16:47:52.9444454Z }, 2024-11-01T16:47:52.9444723Z { 2024-11-01T16:47:52.9445192Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9445792Z "size": 473, 2024-11-01T16:47:52.9446349Z "digest": "sha256:a2855daa5866bf5c1f821fe15ab3dc94d0e04c474534cc8fd61e37dd033f6d21" 2024-11-01T16:47:52.9447023Z }, 2024-11-01T16:47:52.9447259Z { 2024-11-01T16:47:52.9447712Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9448311Z "size": 1374858861, 2024-11-01T16:47:52.9448869Z "digest": "sha256:f1115284e86f76b9b40474519035f12dc2b7a38d77f1a4191f8dce8598c5d862" 2024-11-01T16:47:52.9449504Z }, 2024-11-01T16:47:52.9449735Z { 2024-11-01T16:47:52.9450171Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9450737Z "size": 106, 2024-11-01T16:47:52.9451293Z "digest": "sha256:cb5a67f63c8d177a1786a01d1bb472e22dc3d7602651efef0cec571b39e53994" 2024-11-01T16:47:52.9451945Z }, 2024-11-01T16:47:52.9452176Z { 2024-11-01T16:47:52.9452610Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9453185Z "size": 569, 2024-11-01T16:47:52.9453752Z "digest": "sha256:6c8a5adabac70fa38113960321e02a1be6573fbbd961e2282252361395bf9fda" 2024-11-01T16:47:52.9454394Z }, 2024-11-01T16:47:52.9454631Z { 2024-11-01T16:47:52.9455068Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9455654Z "size": 46248469, 2024-11-01T16:47:52.9456284Z "digest": "sha256:6d9d081053f05d9dd4a153d7530a71014be9e585a52e40da21c7df749ef606e0" 2024-11-01T16:47:52.9456929Z }, 2024-11-01T16:47:52.9457163Z { 2024-11-01T16:47:52.9457624Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9458212Z "size": 111, 2024-11-01T16:47:52.9458803Z "digest": "sha256:cfe7a8ea01eae45e4beaa69ef835009b4ee79c1cc9ca6d6cfa995a37b37d3df9" 2024-11-01T16:47:52.9459459Z }, 2024-11-01T16:47:52.9459705Z { 2024-11-01T16:47:52.9460137Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9460726Z "size": 530, 2024-11-01T16:47:52.9461315Z "digest": "sha256:8031de538f389b95fc35ef1050da7a640fb08a0164bb2e8529eecc841264e3fe" 2024-11-01T16:47:52.9461964Z }, 2024-11-01T16:47:52.9462194Z { 2024-11-01T16:47:52.9462638Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9463232Z "size": 48289019, 2024-11-01T16:47:52.9463829Z "digest": "sha256:e0b62a3e44570cacd74dc71155faa84f674f0f52a5b8eb318f52eeffb1c90df6" 2024-11-01T16:47:52.9464496Z }, 2024-11-01T16:47:52.9464715Z { 2024-11-01T16:47:52.9465155Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9465728Z "size": 106, 2024-11-01T16:47:52.9466287Z "digest": "sha256:14779d140ab7c65e9d97ff2c4a2ebbe64d96396a6f43da515554cd80e4f60313" 2024-11-01T16:47:52.9466942Z }, 2024-11-01T16:47:52.9467173Z { 2024-11-01T16:47:52.9467597Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9468165Z "size": 32, 2024-11-01T16:47:52.9468726Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-11-01T16:47:52.9469377Z }, 2024-11-01T16:47:52.9469608Z { 2024-11-01T16:47:52.9470029Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9470597Z "size": 32, 2024-11-01T16:47:52.9471151Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-11-01T16:47:52.9471800Z }, 2024-11-01T16:47:52.9472038Z { 2024-11-01T16:47:52.9472480Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9473066Z "size": 32, 2024-11-01T16:47:52.9473637Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-11-01T16:47:52.9474370Z }, 2024-11-01T16:47:52.9474629Z { 2024-11-01T16:47:52.9475056Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-11-01T16:47:52.9475662Z "size": 32, 2024-11-01T16:47:52.9476378Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-11-01T16:47:52.9477044Z } 2024-11-01T16:47:52.9477268Z ] 2024-11-01T16:47:52.9477486Z } 2024-11-01T16:47:52.9592088Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-11-01T16:47:52.9592554Z tag=${ECR_DOCKER_IMAGE##*/} 2024-11-01T16:47:52.9593103Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-11-01T16:47:52.9598543Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:52.9599080Z env: 2024-11-01T16:47:52.9599360Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:52.9600728Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.9602138Z ##[endgroup] 2024-11-01T16:47:52.9628031Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.9695715Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-11-01T16:47:52.9696306Z with: 2024-11-01T16:47:52.9697492Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.9698953Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.9699590Z env: 2024-11-01T16:47:52.9699863Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:52.9700213Z ##[endgroup] 2024-11-01T16:47:52.9719330Z ##[group]Run set -x 2024-11-01T16:47:52.9719719Z set -x 2024-11-01T16:47:52.9720021Z set +e 2024-11-01T16:47:52.9720327Z  2024-11-01T16:47:52.9720616Z login() { 2024-11-01T16:47:52.9721365Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-11-01T16:47:52.9722254Z } 2024-11-01T16:47:52.9722504Z  2024-11-01T16:47:52.9722814Z retry () { 2024-11-01T16:47:52.9723204Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-11-01T16:47:52.9723646Z } 2024-11-01T16:47:52.9723912Z  2024-11-01T16:47:52.9724225Z retry login "${DOCKER_REGISTRY}" 2024-11-01T16:47:52.9724648Z  2024-11-01T16:47:52.9724908Z set -e 2024-11-01T16:47:52.9725373Z # ignore output since only exit code is used for conditional 2024-11-01T16:47:52.9726115Z # only pull docker image if it's not available locally 2024-11-01T16:47:52.9726915Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-11-01T16:47:52.9727643Z  retry docker pull "${DOCKER_IMAGE}" 2024-11-01T16:47:52.9728084Z fi 2024-11-01T16:47:52.9733182Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:47:52.9733785Z env: 2024-11-01T16:47:52.9734082Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:47:52.9735678Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:52.9737272Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.9737834Z ##[endgroup] 2024-11-01T16:47:52.9759714Z + set +e 2024-11-01T16:47:52.9761405Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.9763395Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:52.9764592Z + aws ecr get-login-password --region us-east-1 2024-11-01T16:47:52.9765488Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-11-01T16:47:53.4805830Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-11-01T16:47:53.4806798Z Configure a credential helper to remove this warning. See 2024-11-01T16:47:53.4807778Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-11-01T16:47:53.4808438Z 2024-11-01T16:47:53.4808580Z Login Succeeded 2024-11-01T16:47:53.4827884Z + set -e 2024-11-01T16:47:53.4832183Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:53.4971714Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:53.4974386Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:47:53.7221241Z bdd298b12da59246147f016e0693ffd722419941: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-11-01T16:47:53.7224295Z 7a2c55901189: Pulling fs layer 2024-11-01T16:47:53.7225411Z 224fe954d725: Pulling fs layer 2024-11-01T16:47:53.7226945Z 75722010b82e: Pulling fs layer 2024-11-01T16:47:53.7228001Z d527cbbb87e3: Pulling fs layer 2024-11-01T16:47:53.7229104Z b57676e46aee: Pulling fs layer 2024-11-01T16:47:53.7230236Z a8c1e85b5e14: Pulling fs layer 2024-11-01T16:47:53.7230708Z a41a8d1c11c8: Pulling fs layer 2024-11-01T16:47:53.7231165Z 0c1227890755: Pulling fs layer 2024-11-01T16:47:53.7231707Z d8d1234baab3: Pulling fs layer 2024-11-01T16:47:53.7232300Z 7ed32bc8e469: Pulling fs layer 2024-11-01T16:47:53.7232755Z ec1e7978c1fe: Pulling fs layer 2024-11-01T16:47:53.7233214Z 671dad96a4d7: Pulling fs layer 2024-11-01T16:47:53.7233668Z ea0146d98a65: Pulling fs layer 2024-11-01T16:47:53.7234087Z 68392a173946: Pulling fs layer 2024-11-01T16:47:53.7234523Z da40e3f2b2b4: Pulling fs layer 2024-11-01T16:47:53.7234968Z 0794764317b2: Pulling fs layer 2024-11-01T16:47:53.7235401Z 9652f7171409: Pulling fs layer 2024-11-01T16:47:53.7235840Z a65906210ebf: Pulling fs layer 2024-11-01T16:47:53.7236257Z 216dc81274cf: Pulling fs layer 2024-11-01T16:47:53.7236695Z 68ddca485341: Pulling fs layer 2024-11-01T16:47:53.7237139Z cc6151119a1f: Pulling fs layer 2024-11-01T16:47:53.7237578Z 3f19dbe230a9: Pulling fs layer 2024-11-01T16:47:53.7238022Z 29115f672c89: Pulling fs layer 2024-11-01T16:47:53.7238461Z d713d505e57e: Pulling fs layer 2024-11-01T16:47:53.7238879Z 668e2ec2d1a6: Pulling fs layer 2024-11-01T16:47:53.7239321Z b30b6fc0415d: Pulling fs layer 2024-11-01T16:47:53.7239768Z e3f18d02cbdf: Pulling fs layer 2024-11-01T16:47:53.7240226Z 399a6aa7d4a7: Pulling fs layer 2024-11-01T16:47:53.7240670Z 7f31c9564805: Pulling fs layer 2024-11-01T16:47:53.7241070Z d527cbbb87e3: Waiting 2024-11-01T16:47:53.7241464Z 58666e413ec3: Pulling fs layer 2024-11-01T16:47:53.7241891Z d8d1234baab3: Waiting 2024-11-01T16:47:53.7242292Z 4f4eab9b0a12: Pulling fs layer 2024-11-01T16:47:53.7242725Z b57676e46aee: Waiting 2024-11-01T16:47:53.7243089Z 435d64b6b004: Pulling fs layer 2024-11-01T16:47:53.7243541Z 1dc2a0d3d3ba: Pulling fs layer 2024-11-01T16:47:53.7243974Z a8c1e85b5e14: Waiting 2024-11-01T16:47:53.7244445Z 6d4c8df84cee: Pulling fs layer 2024-11-01T16:47:53.7244876Z da40e3f2b2b4: Waiting 2024-11-01T16:47:53.7245277Z b65cd1b6276b: Pulling fs layer 2024-11-01T16:47:53.7245696Z ee2520d1a68e: Pulling fs layer 2024-11-01T16:47:53.7246146Z e9a23ac9bc63: Pulling fs layer 2024-11-01T16:47:53.7246614Z 5bf4f6bf261c: Pulling fs layer 2024-11-01T16:47:53.7247076Z 1c82010b28c5: Pulling fs layer 2024-11-01T16:47:53.7247543Z 6d6440d71b36: Pulling fs layer 2024-11-01T16:47:53.7247986Z a41a8d1c11c8: Waiting 2024-11-01T16:47:53.7248345Z 0794764317b2: Waiting 2024-11-01T16:47:53.7248754Z 4ef0ae975192: Pulling fs layer 2024-11-01T16:47:53.7249214Z 783687f05ffa: Pulling fs layer 2024-11-01T16:47:53.7249653Z 9652f7171409: Waiting 2024-11-01T16:47:53.7250064Z a76f47476cd0: Pulling fs layer 2024-11-01T16:47:53.7250475Z 0c1227890755: Waiting 2024-11-01T16:47:53.7250888Z 71aac5529f94: Pulling fs layer 2024-11-01T16:47:53.7251332Z a65906210ebf: Waiting 2024-11-01T16:47:53.7251725Z 7ed32bc8e469: Waiting 2024-11-01T16:47:53.7252125Z 4f4eab9b0a12: Waiting 2024-11-01T16:47:53.7252510Z fa02b3beb343: Pulling fs layer 2024-11-01T16:47:53.7252963Z 6d4c8df84cee: Waiting 2024-11-01T16:47:53.7253361Z 216dc81274cf: Waiting 2024-11-01T16:47:53.7253761Z 51910ec2cd8f: Pulling fs layer 2024-11-01T16:47:53.7254205Z 435d64b6b004: Waiting 2024-11-01T16:47:53.7254568Z b65cd1b6276b: Waiting 2024-11-01T16:47:53.7254960Z 1dc2a0d3d3ba: Waiting 2024-11-01T16:47:53.7255384Z 0fac4ecd0f3e: Pulling fs layer 2024-11-01T16:47:53.7255825Z 68ddca485341: Waiting 2024-11-01T16:47:53.7256226Z 402d8839fdba: Pulling fs layer 2024-11-01T16:47:53.7256642Z ec1e7978c1fe: Waiting 2024-11-01T16:47:53.7257039Z cc6151119a1f: Waiting 2024-11-01T16:47:53.7257444Z af8723b4a86f: Pulling fs layer 2024-11-01T16:47:53.7257890Z 58666e413ec3: Waiting 2024-11-01T16:47:53.7258278Z d713d505e57e: Waiting 2024-11-01T16:47:53.7258652Z 7389530cf6bd: Pulling fs layer 2024-11-01T16:47:53.7259096Z 671dad96a4d7: Waiting 2024-11-01T16:47:53.7259484Z 3f19dbe230a9: Waiting 2024-11-01T16:47:53.7259992Z 668e2ec2d1a6: Waiting 2024-11-01T16:47:53.7260401Z e138b7611c1c: Pulling fs layer 2024-11-01T16:47:53.7260818Z ea0146d98a65: Waiting 2024-11-01T16:47:53.7261208Z ee2520d1a68e: Waiting 2024-11-01T16:47:53.7261619Z 7449ddbd03b4: Pulling fs layer 2024-11-01T16:47:53.7262116Z dc59e6876e81: Pulling fs layer 2024-11-01T16:47:53.7262567Z e3f18d02cbdf: Waiting 2024-11-01T16:47:53.7262951Z 3ccf55ddb428: Pulling fs layer 2024-11-01T16:47:53.7263470Z 29115f672c89: Waiting 2024-11-01T16:47:53.7263884Z 4f4fb700ef54: Pulling fs layer 2024-11-01T16:47:53.7264336Z 68392a173946: Waiting 2024-11-01T16:47:53.7264721Z 71aac5529f94: Waiting 2024-11-01T16:47:53.7265092Z 7367c224890d: Pulling fs layer 2024-11-01T16:47:53.7265547Z 399a6aa7d4a7: Waiting 2024-11-01T16:47:53.7265924Z fa02b3beb343: Waiting 2024-11-01T16:47:53.7266318Z 9ea471a9650b: Pulling fs layer 2024-11-01T16:47:53.7266744Z b30b6fc0415d: Waiting 2024-11-01T16:47:53.7267113Z 5ac6db3b2d60: Pulling fs layer 2024-11-01T16:47:53.7267548Z 783687f05ffa: Waiting 2024-11-01T16:47:53.7267922Z 7449ddbd03b4: Waiting 2024-11-01T16:47:53.7268302Z 51910ec2cd8f: Waiting 2024-11-01T16:47:53.7268693Z 0eb949a27711: Pulling fs layer 2024-11-01T16:47:53.7269092Z a76f47476cd0: Waiting 2024-11-01T16:47:53.7269483Z b9ba726a0856: Pulling fs layer 2024-11-01T16:47:53.7269915Z dc59e6876e81: Waiting 2024-11-01T16:47:53.7270298Z 0fac4ecd0f3e: Waiting 2024-11-01T16:47:53.7270694Z 9760655c3b12: Pulling fs layer 2024-11-01T16:47:53.7271117Z 62a8fbb6691e: Pulling fs layer 2024-11-01T16:47:53.7271540Z 402d8839fdba: Waiting 2024-11-01T16:47:53.7271931Z e7e230f57c57: Pulling fs layer 2024-11-01T16:47:53.7272377Z 10e81741e898: Pulling fs layer 2024-11-01T16:47:53.7272823Z b1aaa169430e: Pulling fs layer 2024-11-01T16:47:53.7273240Z a2855daa5866: Pulling fs layer 2024-11-01T16:47:53.7273684Z f1115284e86f: Pulling fs layer 2024-11-01T16:47:53.7274127Z cb5a67f63c8d: Pulling fs layer 2024-11-01T16:47:53.7274577Z 6c8a5adabac7: Pulling fs layer 2024-11-01T16:47:53.7275011Z 5ac6db3b2d60: Waiting 2024-11-01T16:47:53.7275496Z 6d9d081053f0: Pulling fs layer 2024-11-01T16:47:53.7275912Z 0eb949a27711: Waiting 2024-11-01T16:47:53.7276308Z cfe7a8ea01ea: Pulling fs layer 2024-11-01T16:47:53.7276755Z 8031de538f38: Pulling fs layer 2024-11-01T16:47:53.7277170Z af8723b4a86f: Waiting 2024-11-01T16:47:53.7277530Z e0b62a3e4457: Pulling fs layer 2024-11-01T16:47:53.7277946Z 1c82010b28c5: Waiting 2024-11-01T16:47:53.7278328Z 14779d140ab7: Pulling fs layer 2024-11-01T16:47:53.7278747Z 6d6440d71b36: Waiting 2024-11-01T16:47:53.7279112Z 4f4fb700ef54: Waiting 2024-11-01T16:47:53.7279449Z 7367c224890d: Waiting 2024-11-01T16:47:53.7279807Z 9ea471a9650b: Waiting 2024-11-01T16:47:53.7280173Z 7f31c9564805: Waiting 2024-11-01T16:47:53.7280537Z b9ba726a0856: Waiting 2024-11-01T16:47:53.7280898Z e138b7611c1c: Waiting 2024-11-01T16:47:53.7281234Z e0b62a3e4457: Waiting 2024-11-01T16:47:53.7281594Z 9760655c3b12: Waiting 2024-11-01T16:47:53.7281965Z cfe7a8ea01ea: Waiting 2024-11-01T16:47:53.7282340Z b1aaa169430e: Waiting 2024-11-01T16:47:53.7282706Z 8031de538f38: Waiting 2024-11-01T16:47:53.7283046Z e7e230f57c57: Waiting 2024-11-01T16:47:53.7283411Z 62a8fbb6691e: Waiting 2024-11-01T16:47:53.7283787Z a2855daa5866: Waiting 2024-11-01T16:47:53.7284156Z f1115284e86f: Waiting 2024-11-01T16:47:53.7284501Z 6d9d081053f0: Waiting 2024-11-01T16:47:53.7284822Z 10e81741e898: Waiting 2024-11-01T16:47:53.8448644Z 224fe954d725: Verifying Checksum 2024-11-01T16:47:53.8449187Z 224fe954d725: Download complete 2024-11-01T16:47:53.9075332Z d527cbbb87e3: Download complete 2024-11-01T16:47:53.9885656Z b57676e46aee: Verifying Checksum 2024-11-01T16:47:53.9886281Z b57676e46aee: Download complete 2024-11-01T16:47:54.0703156Z 7a2c55901189: Download complete 2024-11-01T16:47:54.1588194Z a41a8d1c11c8: Download complete 2024-11-01T16:47:54.2227147Z 0c1227890755: Verifying Checksum 2024-11-01T16:47:54.2227734Z 0c1227890755: Download complete 2024-11-01T16:47:54.2928096Z d8d1234baab3: Verifying Checksum 2024-11-01T16:47:54.2928648Z d8d1234baab3: Download complete 2024-11-01T16:47:54.3273905Z 75722010b82e: Verifying Checksum 2024-11-01T16:47:54.3274431Z 75722010b82e: Download complete 2024-11-01T16:47:54.4166186Z ec1e7978c1fe: Verifying Checksum 2024-11-01T16:47:54.4166749Z ec1e7978c1fe: Download complete 2024-11-01T16:47:54.4735716Z 671dad96a4d7: Download complete 2024-11-01T16:47:54.8440146Z 7a2c55901189: Pull complete 2024-11-01T16:47:55.0624990Z 224fe954d725: Pull complete 2024-11-01T16:47:55.8611136Z 75722010b82e: Pull complete 2024-11-01T16:47:55.8851346Z d527cbbb87e3: Pull complete 2024-11-01T16:47:55.9066181Z b57676e46aee: Pull complete 2024-11-01T16:47:56.9887657Z ea0146d98a65: Verifying Checksum 2024-11-01T16:47:56.9888973Z ea0146d98a65: Download complete 2024-11-01T16:47:57.0549535Z 68392a173946: Download complete 2024-11-01T16:47:57.1255436Z da40e3f2b2b4: Download complete 2024-11-01T16:47:57.1958063Z 0794764317b2: Verifying Checksum 2024-11-01T16:47:57.1958616Z 0794764317b2: Download complete 2024-11-01T16:47:58.1740601Z 9652f7171409: Verifying Checksum 2024-11-01T16:47:58.1741256Z 9652f7171409: Download complete 2024-11-01T16:47:58.2465679Z a65906210ebf: Download complete 2024-11-01T16:47:58.3254577Z 216dc81274cf: Download complete 2024-11-01T16:47:58.4019581Z 68ddca485341: Download complete 2024-11-01T16:48:07.3331149Z a8c1e85b5e14: Verifying Checksum 2024-11-01T16:48:07.3331726Z a8c1e85b5e14: Download complete 2024-11-01T16:48:07.4066117Z 3f19dbe230a9: Verifying Checksum 2024-11-01T16:48:07.4066696Z 3f19dbe230a9: Download complete 2024-11-01T16:48:07.4874560Z 29115f672c89: Verifying Checksum 2024-11-01T16:48:07.4875105Z 29115f672c89: Download complete 2024-11-01T16:48:07.5620853Z d713d505e57e: Verifying Checksum 2024-11-01T16:48:07.6581103Z 668e2ec2d1a6: Download complete 2024-11-01T16:48:07.7330462Z b30b6fc0415d: Verifying Checksum 2024-11-01T16:48:07.7331040Z b30b6fc0415d: Download complete 2024-11-01T16:48:09.0114703Z e3f18d02cbdf: Verifying Checksum 2024-11-01T16:48:09.0116233Z e3f18d02cbdf: Download complete 2024-11-01T16:48:09.0865625Z 399a6aa7d4a7: Verifying Checksum 2024-11-01T16:48:09.0866188Z 399a6aa7d4a7: Download complete 2024-11-01T16:48:09.1629867Z 7f31c9564805: Download complete 2024-11-01T16:48:09.2423988Z 58666e413ec3: Verifying Checksum 2024-11-01T16:48:09.2424679Z 58666e413ec3: Download complete 2024-11-01T16:48:09.3166350Z 4f4eab9b0a12: Download complete 2024-11-01T16:48:09.3901690Z 435d64b6b004: Download complete 2024-11-01T16:48:13.7174145Z 1dc2a0d3d3ba: Verifying Checksum 2024-11-01T16:48:13.7174789Z 1dc2a0d3d3ba: Download complete 2024-11-01T16:48:13.8468252Z 6d4c8df84cee: Download complete 2024-11-01T16:48:13.9139965Z b65cd1b6276b: Verifying Checksum 2024-11-01T16:48:13.9140538Z b65cd1b6276b: Download complete 2024-11-01T16:48:14.3213346Z ee2520d1a68e: Verifying Checksum 2024-11-01T16:48:14.3213880Z ee2520d1a68e: Download complete 2024-11-01T16:48:14.3882315Z e9a23ac9bc63: Verifying Checksum 2024-11-01T16:48:14.3882915Z e9a23ac9bc63: Download complete 2024-11-01T16:48:14.4922012Z 5bf4f6bf261c: Verifying Checksum 2024-11-01T16:48:14.4922613Z 5bf4f6bf261c: Download complete 2024-11-01T16:48:14.7527245Z 1c82010b28c5: Verifying Checksum 2024-11-01T16:48:14.7527852Z 1c82010b28c5: Download complete 2024-11-01T16:48:14.8232783Z 6d6440d71b36: Verifying Checksum 2024-11-01T16:48:14.8234014Z 6d6440d71b36: Download complete 2024-11-01T16:48:14.8990874Z 4ef0ae975192: Download complete 2024-11-01T16:48:15.0503100Z 783687f05ffa: Verifying Checksum 2024-11-01T16:48:15.0503729Z 783687f05ffa: Download complete 2024-11-01T16:48:19.3597197Z a8c1e85b5e14: Pull complete 2024-11-01T16:48:19.6211215Z 7ed32bc8e469: Verifying Checksum 2024-11-01T16:48:19.6211833Z 7ed32bc8e469: Download complete 2024-11-01T16:48:19.6969520Z 71aac5529f94: Verifying Checksum 2024-11-01T16:48:19.6970822Z 71aac5529f94: Download complete 2024-11-01T16:48:19.7722672Z fa02b3beb343: Verifying Checksum 2024-11-01T16:48:19.7723225Z fa02b3beb343: Download complete 2024-11-01T16:48:19.7965809Z a41a8d1c11c8: Pull complete 2024-11-01T16:48:20.2170639Z 0c1227890755: Pull complete 2024-11-01T16:48:20.3467259Z 51910ec2cd8f: Verifying Checksum 2024-11-01T16:48:20.3467853Z 51910ec2cd8f: Download complete 2024-11-01T16:48:20.4280316Z 0fac4ecd0f3e: Verifying Checksum 2024-11-01T16:48:20.4280982Z 0fac4ecd0f3e: Download complete 2024-11-01T16:48:20.5053267Z 402d8839fdba: Download complete 2024-11-01T16:48:20.5889354Z af8723b4a86f: Download complete 2024-11-01T16:48:20.6532941Z 7389530cf6bd: Verifying Checksum 2024-11-01T16:48:20.6534595Z 7389530cf6bd: Download complete 2024-11-01T16:48:20.6804399Z d8d1234baab3: Pull complete 2024-11-01T16:48:27.3805341Z cc6151119a1f: Verifying Checksum 2024-11-01T16:48:27.3805937Z cc6151119a1f: Download complete 2024-11-01T16:48:27.4618865Z 7449ddbd03b4: Verifying Checksum 2024-11-01T16:48:27.4619439Z 7449ddbd03b4: Download complete 2024-11-01T16:48:27.5585191Z dc59e6876e81: Verifying Checksum 2024-11-01T16:48:27.5585775Z dc59e6876e81: Download complete 2024-11-01T16:48:27.6378605Z 3ccf55ddb428: Verifying Checksum 2024-11-01T16:48:27.6379195Z 3ccf55ddb428: Download complete 2024-11-01T16:48:27.6468635Z 4f4fb700ef54: Verifying Checksum 2024-11-01T16:48:27.6469269Z 4f4fb700ef54: Download complete 2024-11-01T16:48:27.7096230Z 7367c224890d: Download complete 2024-11-01T16:48:27.7898705Z 9ea471a9650b: Verifying Checksum 2024-11-01T16:48:27.7899334Z 9ea471a9650b: Download complete 2024-11-01T16:48:27.9100675Z 5ac6db3b2d60: Verifying Checksum 2024-11-01T16:48:27.9101296Z 5ac6db3b2d60: Download complete 2024-11-01T16:48:27.9847400Z 0eb949a27711: Download complete 2024-11-01T16:48:28.0614359Z b9ba726a0856: Download complete 2024-11-01T16:48:28.1348370Z 9760655c3b12: Download complete 2024-11-01T16:48:28.2354576Z 62a8fbb6691e: Verifying Checksum 2024-11-01T16:48:28.2355205Z 62a8fbb6691e: Download complete 2024-11-01T16:48:28.3685046Z e7e230f57c57: Verifying Checksum 2024-11-01T16:48:28.3685634Z e7e230f57c57: Download complete 2024-11-01T16:48:28.4492002Z 10e81741e898: Verifying Checksum 2024-11-01T16:48:28.4492576Z 10e81741e898: Download complete 2024-11-01T16:48:29.0273268Z b1aaa169430e: Verifying Checksum 2024-11-01T16:48:29.0273848Z b1aaa169430e: Download complete 2024-11-01T16:48:29.1134869Z a2855daa5866: Verifying Checksum 2024-11-01T16:48:29.1135466Z a2855daa5866: Download complete 2024-11-01T16:48:42.9187278Z f1115284e86f: Verifying Checksum 2024-11-01T16:48:42.9187861Z f1115284e86f: Download complete 2024-11-01T16:48:43.0087515Z cb5a67f63c8d: Download complete 2024-11-01T16:48:43.1107622Z 6c8a5adabac7: Verifying Checksum 2024-11-01T16:48:43.1108161Z 6c8a5adabac7: Download complete 2024-11-01T16:48:43.6259398Z 6d9d081053f0: Verifying Checksum 2024-11-01T16:48:43.6259964Z 6d9d081053f0: Download complete 2024-11-01T16:48:43.7049123Z cfe7a8ea01ea: Verifying Checksum 2024-11-01T16:48:43.7049745Z cfe7a8ea01ea: Download complete 2024-11-01T16:48:43.7800161Z 8031de538f38: Download complete 2024-11-01T16:48:44.3122984Z e0b62a3e4457: Verifying Checksum 2024-11-01T16:48:44.3123583Z e0b62a3e4457: Download complete 2024-11-01T16:48:44.3870626Z 14779d140ab7: Verifying Checksum 2024-11-01T16:48:44.3871219Z 14779d140ab7: Download complete 2024-11-01T16:48:46.9477370Z 7ed32bc8e469: Pull complete 2024-11-01T16:48:47.4052407Z ec1e7978c1fe: Pull complete 2024-11-01T16:48:47.8801192Z 671dad96a4d7: Pull complete 2024-11-01T16:48:54.6938405Z ea0146d98a65: Pull complete 2024-11-01T16:48:55.1473431Z 68392a173946: Pull complete 2024-11-01T16:48:55.6397235Z da40e3f2b2b4: Pull complete 2024-11-01T16:48:56.1106008Z 0794764317b2: Pull complete 2024-11-01T16:48:58.4767487Z 9652f7171409: Pull complete 2024-11-01T16:48:58.9223269Z a65906210ebf: Pull complete 2024-11-01T16:48:59.4122200Z 216dc81274cf: Pull complete 2024-11-01T16:48:59.8740215Z 68ddca485341: Pull complete 2024-11-01T16:49:03.6994494Z e138b7611c1c: Verifying Checksum 2024-11-01T16:49:03.6995101Z e138b7611c1c: Download complete 2024-11-01T16:49:05.4629610Z a76f47476cd0: Verifying Checksum 2024-11-01T16:49:05.4630341Z a76f47476cd0: Download complete 2024-11-01T16:49:44.6726577Z cc6151119a1f: Pull complete 2024-11-01T16:49:45.1210264Z 3f19dbe230a9: Pull complete 2024-11-01T16:49:45.5840045Z 29115f672c89: Pull complete 2024-11-01T16:49:46.0530712Z d713d505e57e: Pull complete 2024-11-01T16:49:46.5105140Z 668e2ec2d1a6: Pull complete 2024-11-01T16:49:46.9408178Z b30b6fc0415d: Pull complete 2024-11-01T16:49:50.1032890Z e3f18d02cbdf: Pull complete 2024-11-01T16:49:50.4429052Z 399a6aa7d4a7: Pull complete 2024-11-01T16:49:50.5512489Z 7f31c9564805: Pull complete 2024-11-01T16:49:50.8126221Z 58666e413ec3: Pull complete 2024-11-01T16:49:51.2834903Z 4f4eab9b0a12: Pull complete 2024-11-01T16:49:51.6709889Z 435d64b6b004: Pull complete 2024-11-01T16:49:59.1725219Z 1dc2a0d3d3ba: Pull complete 2024-11-01T16:49:59.5870859Z 6d4c8df84cee: Pull complete 2024-11-01T16:49:59.8588301Z b65cd1b6276b: Pull complete 2024-11-01T16:50:00.8856125Z ee2520d1a68e: Pull complete 2024-11-01T16:50:01.3059781Z e9a23ac9bc63: Pull complete 2024-11-01T16:50:01.6803817Z 5bf4f6bf261c: Pull complete 2024-11-01T16:50:02.4245330Z 1c82010b28c5: Pull complete 2024-11-01T16:50:02.8645440Z 6d6440d71b36: Pull complete 2024-11-01T16:50:03.7672019Z 4ef0ae975192: Pull complete 2024-11-01T16:50:04.2021884Z 783687f05ffa: Pull complete 2024-11-01T16:50:40.0918199Z a76f47476cd0: Pull complete 2024-11-01T16:50:40.5479206Z 71aac5529f94: Pull complete 2024-11-01T16:50:40.9933585Z fa02b3beb343: Pull complete 2024-11-01T16:50:43.1187111Z 51910ec2cd8f: Pull complete 2024-11-01T16:50:43.5693762Z 0fac4ecd0f3e: Pull complete 2024-11-01T16:50:44.0038095Z 402d8839fdba: Pull complete 2024-11-01T16:50:44.6803036Z af8723b4a86f: Pull complete 2024-11-01T16:50:45.0836328Z 7389530cf6bd: Pull complete 2024-11-01T16:51:57.3858559Z e138b7611c1c: Pull complete 2024-11-01T16:51:57.8417607Z 7449ddbd03b4: Pull complete 2024-11-01T16:51:58.2927263Z dc59e6876e81: Pull complete 2024-11-01T16:51:59.1988943Z 3ccf55ddb428: Pull complete 2024-11-01T16:51:59.6301803Z 4f4fb700ef54: Pull complete 2024-11-01T16:52:00.0123248Z 7367c224890d: Pull complete 2024-11-01T16:52:00.2500939Z 9ea471a9650b: Pull complete 2024-11-01T16:52:00.6885488Z 5ac6db3b2d60: Pull complete 2024-11-01T16:52:01.1317660Z 0eb949a27711: Pull complete 2024-11-01T16:52:01.5890477Z b9ba726a0856: Pull complete 2024-11-01T16:52:01.9857325Z 9760655c3b12: Pull complete 2024-11-01T16:52:02.4301168Z 62a8fbb6691e: Pull complete 2024-11-01T16:52:03.6497993Z e7e230f57c57: Pull complete 2024-11-01T16:52:03.9498542Z 10e81741e898: Pull complete 2024-11-01T16:52:05.8659958Z b1aaa169430e: Pull complete 2024-11-01T16:52:06.2257707Z a2855daa5866: Pull complete 2024-11-01T16:52:19.1010468Z f1115284e86f: Pull complete 2024-11-01T16:52:19.5134995Z cb5a67f63c8d: Pull complete 2024-11-01T16:52:19.9639871Z 6c8a5adabac7: Pull complete 2024-11-01T16:52:21.0060007Z 6d9d081053f0: Pull complete 2024-11-01T16:52:21.3992132Z cfe7a8ea01ea: Pull complete 2024-11-01T16:52:21.7992770Z 8031de538f38: Pull complete 2024-11-01T16:52:22.6365001Z e0b62a3e4457: Pull complete 2024-11-01T16:52:23.0801092Z 14779d140ab7: Pull complete 2024-11-01T16:52:25.1230432Z Digest: sha256:cc02ebc4db7e1029fdcc4fd31c0bc7bc340781604fac6b0edc559fbcd1ec89c4 2024-11-01T16:52:25.2078568Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:52:25.2423334Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:52:25.2496781Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2024-11-01T16:52:25.2498341Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2024-11-01T16:52:25.2507729Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:52:25.2508346Z env: 2024-11-01T16:52:25.2508694Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:52:25.2509129Z ##[endgroup] 2024-11-01T16:52:25.2728915Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-11-01T16:52:25.2729502Z with: 2024-11-01T16:52:25.2729781Z driver-version: 550.54.15 2024-11-01T16:52:25.2730147Z env: 2024-11-01T16:52:25.2730436Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:52:25.2730799Z ##[endgroup] 2024-11-01T16:52:25.3003814Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-11-01T16:52:25.3004443Z with: 2024-11-01T16:52:25.3005032Z timeout_minutes: 10 2024-11-01T16:52:25.3005368Z max_attempts: 3 2024-11-01T16:52:25.3045468Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-11-01T16:52:25.3084161Z retry_wait_seconds: 10 2024-11-01T16:52:25.3084542Z polling_interval_seconds: 1 2024-11-01T16:52:25.3084941Z warning_on_retry: true 2024-11-01T16:52:25.3085302Z continue_on_error: false 2024-11-01T16:52:25.3085639Z env: 2024-11-01T16:52:25.3085922Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:52:25.3086295Z DRIVER_VERSION: 550.54.15 2024-11-01T16:52:25.3086660Z ##[endgroup] 2024-11-01T16:52:25.3849929Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-11-01T16:52:25.3851145Z + pre_install_nvidia_driver_amzn2 2024-11-01T16:52:25.3852992Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-11-01T16:52:25.7166906Z No match for argument: nvidia-driver-latest-dkms 2024-11-01T16:52:25.7167598Z No packages marked for removal. 2024-11-01T16:52:25.7234429Z Dependencies resolved. 2024-11-01T16:52:25.7244555Z Nothing to do. 2024-11-01T16:52:25.7244871Z Complete! 2024-11-01T16:52:25.8312875Z + install_nvidia_driver_common 2024-11-01T16:52:25.8314739Z + echo 'Before installing NVIDIA driver' 2024-11-01T16:52:25.8315370Z + lspci 2024-11-01T16:52:25.8315799Z Before installing NVIDIA driver 2024-11-01T16:52:25.8540467Z 00:00.0 Host bridge: Intel Corporation Sky Lake-E DMI3 Registers (rev 07) 2024-11-01T16:52:25.8541714Z 00:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8542786Z 00:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8543851Z 00:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8544922Z 00:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8545976Z 00:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8547018Z 00:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8548061Z 00:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8549351Z 00:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8550544Z 00:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 07) 2024-11-01T16:52:25.8551638Z 00:05.2 System peripheral: Intel Corporation Sky Lake-E RAS (rev 07) 2024-11-01T16:52:25.8552467Z 00:05.4 PIC: Intel Corporation Sky Lake-E IOAPIC (rev 07) 2024-11-01T16:52:25.8553463Z 00:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:25.8554664Z 00:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:25.8555719Z 00:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:25.8556748Z 00:11.0 Unassigned class [ff00]: Intel Corporation C620 Series Chipset Family MROM 0 (rev 09) 2024-11-01T16:52:25.8557843Z 00:11.1 Unassigned class [ff00]: Intel Corporation C620 Series Chipset Family MROM 1 (rev 09) 2024-11-01T16:52:25.8559058Z 00:11.5 SATA controller: Intel Corporation C620 Series Chipset Family SSATA Controller [AHCI mode] (rev 09) 2024-11-01T16:52:25.8560326Z 00:14.0 USB controller: Intel Corporation C620 Series Chipset Family USB 3.0 xHCI Controller (rev 09) 2024-11-01T16:52:25.8561597Z 00:14.2 Signal processing controller: Intel Corporation C620 Series Chipset Family Thermal Subsystem (rev 09) 2024-11-01T16:52:25.8562905Z 00:16.0 Communication controller: Intel Corporation C620 Series Chipset Family MEI Controller #1 (rev 09) 2024-11-01T16:52:25.8564173Z 00:16.4 Communication controller: Intel Corporation C620 Series Chipset Family MEI Controller #3 (rev 09) 2024-11-01T16:52:25.8565456Z 00:17.0 SATA controller: Intel Corporation C620 Series Chipset Family SATA Controller [AHCI mode] (rev 09) 2024-11-01T16:52:25.8566604Z 00:1f.0 ISA bridge: Intel Corporation C621 Series Chipset LPC/eSPI Controller (rev 09) 2024-11-01T16:52:25.8567790Z 00:1f.2 Memory controller: Intel Corporation C620 Series Chipset Family Power Management Controller (rev 09) 2024-11-01T16:52:25.8568900Z 00:1f.4 SMBus: Intel Corporation C620 Series Chipset Family SMBus (rev 09) 2024-11-01T16:52:25.8569943Z 00:1f.5 Serial bus controller: Intel Corporation C620 Series Chipset Family SPI Controller (rev 09) 2024-11-01T16:52:25.8571130Z 17:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:25.8572192Z 17:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:25.8573183Z 17:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:25.8574269Z 17:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:25.8575423Z 17:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:25.8576472Z 17:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8577500Z 17:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8578517Z 17:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8579532Z 17:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8580551Z 17:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8581592Z 17:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8582643Z 17:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8583678Z 17:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8584736Z 17:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8585762Z 17:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8586826Z 17:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8588006Z 17:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8589042Z 17:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8590063Z 17:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8591062Z 17:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8592072Z 17:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8593113Z 17:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8594228Z 17:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8595282Z 17:0a.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8596329Z 17:0a.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8597388Z 17:0a.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8598419Z 17:0a.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8599459Z 17:0a.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8600488Z 17:0a.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8601539Z 17:0b.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8602584Z 17:0b.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8603640Z 17:0b.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8604670Z 17:0b.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8605719Z 17:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8606764Z 17:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8607796Z 17:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8608838Z 17:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8609994Z 17:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8611020Z 17:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8612074Z 17:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8613112Z 17:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8614180Z 17:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8615225Z 17:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8616251Z 17:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8617274Z 17:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8618328Z 17:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8619367Z 17:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8620415Z 17:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8621458Z 17:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8622911Z 17:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8624047Z 17:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8624989Z 17:10.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8625954Z 17:10.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8626915Z 17:10.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8627982Z 17:10.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8628943Z 17:10.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8629904Z 17:10.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8630839Z 17:11.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8631790Z 17:11.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8633006Z 17:11.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8633977Z 17:11.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8634943Z 17:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8636070Z 17:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8637073Z 17:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8638265Z 17:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8639306Z 17:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8640347Z 17:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8641393Z 17:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8642438Z 17:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8643493Z 17:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8644514Z 17:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8645550Z 17:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8646439Z 18:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:25.8647288Z 19:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:25.8648307Z 34:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:25.8649387Z 34:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:25.8650364Z 34:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:25.8651459Z 34:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:25.8652618Z 34:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:25.8653801Z 34:08.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8655021Z 34:09.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8656257Z 34:0a.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8657496Z 34:0a.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8658721Z 34:0a.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8659981Z 34:0a.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8661232Z 34:0a.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8662378Z 34:0a.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07) 2024-11-01T16:52:25.8663440Z 34:0a.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07) 2024-11-01T16:52:25.8664494Z 34:0a.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07) 2024-11-01T16:52:25.8665533Z 34:0b.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07) 2024-11-01T16:52:25.8666581Z 34:0b.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07) 2024-11-01T16:52:25.8667867Z 34:0b.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07) 2024-11-01T16:52:25.8668839Z 34:0b.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07) 2024-11-01T16:52:25.8669916Z 34:0c.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8671064Z 34:0c.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8672222Z 34:0c.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8673419Z 34:0c.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8674567Z 34:0c.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8675620Z 34:0c.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07) 2024-11-01T16:52:25.8676589Z 34:0c.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07) 2024-11-01T16:52:25.8677585Z 34:0c.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07) 2024-11-01T16:52:25.8678561Z 34:0d.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07) 2024-11-01T16:52:25.8679505Z 34:0d.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07) 2024-11-01T16:52:25.8680465Z 34:0d.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07) 2024-11-01T16:52:25.8681437Z 34:0d.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07) 2024-11-01T16:52:25.8682282Z 35:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:25.8683054Z 36:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:25.8684196Z 7a:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:25.8685422Z 7a:01.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port B (rev 07) 2024-11-01T16:52:25.8686511Z 7a:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:25.8687520Z 7a:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:25.8688608Z 7a:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:25.8689778Z 7a:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:25.8690849Z 7a:0e.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:25.8691948Z 7a:0e.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:25.8693080Z 7a:0f.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:25.8694200Z 7a:0f.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:25.8695267Z 7a:10.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:25.8696358Z 7a:10.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:25.8697539Z 7a:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8698682Z 7a:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8699778Z 7a:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8700841Z 7a:12.4 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8701944Z 7a:12.5 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8703041Z 7a:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:25.8704138Z 7a:15.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:25.8705232Z 7a:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:25.8706320Z 7a:16.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:25.8707453Z 7a:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:25.8708572Z 7a:16.5 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:25.8709734Z 7a:17.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:25.8710951Z 7a:17.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:25.8711788Z 7b:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8712413Z 7c:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8713112Z 7c:01.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8713725Z 7c:02.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8714343Z 7c:03.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8714954Z 7c:04.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8715567Z 7c:05.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8716164Z 7c:06.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8716742Z 7c:07.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8717341Z 7c:08.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8717942Z 7c:09.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8718536Z 7c:0a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8719127Z 7c:0b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8719708Z 7c:0c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8720301Z 7c:0d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8720897Z 7c:0e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8721486Z 7c:0f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8722078Z 7c:10.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8722670Z 7c:11.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8723251Z 7c:12.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8723842Z 7c:13.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8724447Z 7c:14.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8725051Z 7c:15.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8725625Z 7c:16.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8726181Z 7c:17.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8726763Z 7c:18.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8727359Z 7c:19.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8727947Z 7c:1a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8728544Z 7c:1b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8729141Z 7c:1c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8729806Z 7c:1d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8730393Z 7c:1e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8730977Z 7c:1f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8731553Z 9d:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8732139Z 9e:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8732703Z 9e:01.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8733284Z 9e:02.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8733869Z 9e:03.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8734570Z 9e:04.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8735127Z 9e:05.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8735681Z 9e:06.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8736214Z 9e:07.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8736760Z 9e:08.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8737302Z 9e:09.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8737822Z 9e:0a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8738348Z 9e:0b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8738856Z 9e:0c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8739491Z 9e:0d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8740028Z 9e:0e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8740571Z 9e:0f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8741110Z 9e:10.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8741638Z 9e:11.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8742155Z 9e:12.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8742718Z 9e:13.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8743312Z 9e:14.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8743865Z 9e:15.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8744422Z 9e:16.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8744948Z 9e:17.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8745499Z 9e:18.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8746049Z 9e:19.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8746605Z 9e:1a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8747169Z 9e:1b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8747722Z 9e:1c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8748253Z 9e:1d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8748810Z 9e:1e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8749364Z 9e:1f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8749961Z 9f:00.0 Serial controller: Amazon.com, Inc. Device 8250 2024-11-01T16:52:25.8750771Z a0:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device 0061 2024-11-01T16:52:25.8751619Z bf:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device cd00 2024-11-01T16:52:25.8752488Z bf:00.1 Non-Volatile memory controller: Amazon.com, Inc. Device cd00 2024-11-01T16:52:25.8753434Z c0:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8754435Z c0:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8755659Z c0:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8756713Z c0:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8757738Z c0:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8758782Z c0:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8759827Z c0:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8760885Z c0:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:25.8762047Z c0:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 07) 2024-11-01T16:52:25.8763126Z c0:05.2 System peripheral: Intel Corporation Sky Lake-E RAS (rev 07) 2024-11-01T16:52:25.8763958Z c0:05.4 PIC: Intel Corporation Sky Lake-E IOAPIC (rev 07) 2024-11-01T16:52:25.8764849Z c0:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:25.8766096Z c0:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:25.8767189Z c0:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:25.8768277Z c2:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:25.8769298Z c2:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:25.8770410Z c2:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:25.8771568Z c2:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:25.8772657Z c2:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8773716Z c2:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8774774Z c2:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8775892Z c2:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8777035Z c2:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8778118Z c2:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8779093Z c2:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8780067Z c2:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8781080Z c2:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8782045Z c2:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8783017Z c2:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8783987Z c2:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8784936Z c2:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8785894Z c2:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8786852Z c2:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8787820Z c2:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8788784Z c2:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8789758Z c2:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8790697Z c2:0a.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8791661Z c2:0a.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8792636Z c2:0a.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8793602Z c2:0a.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8794576Z c2:0a.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8795715Z c2:0a.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8796713Z c2:0b.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8797732Z c2:0b.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8798750Z c2:0b.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8799767Z c2:0b.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8800784Z c2:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8801793Z c2:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8802813Z c2:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8803821Z c2:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8804844Z c2:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8805865Z c2:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8806882Z c2:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8807903Z c2:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8808931Z c2:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8810131Z c2:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8811188Z c2:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8812247Z c2:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8813298Z c2:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8814444Z c2:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8815506Z c2:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8816540Z c2:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8817603Z c2:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8818670Z c2:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8819784Z c2:10.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8820824Z c2:10.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8821873Z c2:10.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8822990Z c2:10.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8824182Z c2:10.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8825228Z c2:10.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8826274Z c2:11.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8827579Z c2:11.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8828625Z c2:11.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8829672Z c2:11.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8830693Z c2:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8831751Z c2:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8832804Z c2:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8833847Z c2:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:25.8834905Z c2:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8835951Z c2:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8836989Z c2:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8838134Z c2:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8839246Z c2:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8840218Z c2:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8841181Z c2:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:25.8841914Z c3:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8842456Z c4:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8843014Z c4:01.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8843581Z c4:02.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8844139Z c4:03.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8844698Z c4:04.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8845427Z c4:05.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8846003Z c4:06.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8846590Z c4:07.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8847359Z c4:08.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8847969Z c4:09.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8848574Z c4:0a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8849160Z c4:0b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8849766Z c4:0c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8850377Z c4:0d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8850986Z c4:0e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8851652Z c4:0f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:25.8852436Z c6:00.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-11-01T16:52:25.8853491Z e6:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:25.8854599Z e6:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:25.8855617Z e6:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:25.8856794Z e6:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:25.8857986Z e6:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:25.8859191Z e6:08.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8860439Z e6:09.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8861717Z e6:0a.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8862981Z e6:0a.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8864242Z e6:0a.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8865498Z e6:0a.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8866758Z e6:0a.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8867913Z e6:0a.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07) 2024-11-01T16:52:25.8868942Z e6:0a.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07) 2024-11-01T16:52:25.8870011Z e6:0a.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07) 2024-11-01T16:52:25.8871074Z e6:0b.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07) 2024-11-01T16:52:25.8872135Z e6:0b.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07) 2024-11-01T16:52:25.8873190Z e6:0b.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07) 2024-11-01T16:52:25.8874259Z e6:0b.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07) 2024-11-01T16:52:25.8875566Z e6:0c.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8876992Z e6:0c.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8878248Z e6:0c.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8879480Z e6:0c.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8880707Z e6:0c.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:25.8881829Z e6:0c.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07) 2024-11-01T16:52:25.8882861Z e6:0c.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07) 2024-11-01T16:52:25.8883882Z e6:0c.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07) 2024-11-01T16:52:25.8884923Z e6:0d.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07) 2024-11-01T16:52:25.8885951Z e6:0d.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07) 2024-11-01T16:52:25.8886975Z e6:0d.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07) 2024-11-01T16:52:25.8888015Z e6:0d.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07) 2024-11-01T16:52:25.8888903Z e7:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:25.8889801Z e8:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:25.8890829Z f3:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:25.8891861Z f3:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:25.8892777Z f3:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:25.8893776Z f3:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:25.8894833Z f3:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:25.8895766Z f3:0e.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:25.8896687Z f3:0e.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:25.8897815Z f3:0f.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:25.8898793Z f3:0f.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:25.8899759Z f3:10.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:25.8900727Z f3:10.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:25.8901761Z f3:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8902974Z f3:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8904056Z f3:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8905132Z f3:12.4 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8906224Z f3:12.5 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:25.8907307Z f3:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:25.8908375Z f3:15.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:25.8909440Z f3:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:25.8910569Z f3:16.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:25.8911644Z f3:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:25.8912718Z f3:16.5 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:25.8913795Z f3:17.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:25.8914862Z f3:17.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:25.8915752Z f4:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:25.8916569Z f5:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:25.8917172Z + lsmod 2024-11-01T16:52:25.8917473Z Module Size Used by 2024-11-01T16:52:25.8917907Z veth 36864 0 2024-11-01T16:52:25.8918316Z nvidia_modeset 1351680 0 2024-11-01T16:52:25.8918758Z video 65536 1 nvidia_modeset 2024-11-01T16:52:25.8919239Z wmi 36864 1 video 2024-11-01T16:52:25.8919663Z nvidia_uvm 4706304 0 2024-11-01T16:52:25.8920167Z nvidia 54071296 35 nvidia_uvm,nvidia_modeset 2024-11-01T16:52:25.8920709Z drm 602112 1 nvidia 2024-11-01T16:52:25.8921181Z drm_panel_orientation_quirks 32768 1 drm 2024-11-01T16:52:25.8921772Z backlight 24576 3 video,drm,nvidia_modeset 2024-11-01T16:52:25.8922337Z i2c_core 106496 2 nvidia,drm 2024-11-01T16:52:25.8923017Z xt_conntrack 16384 1 2024-11-01T16:52:25.8923396Z nft_chain_nat 16384 3 2024-11-01T16:52:25.8923768Z xt_MASQUERADE 20480 1 2024-11-01T16:52:25.8924223Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-11-01T16:52:25.8924722Z nf_conntrack_netlink 57344 0 2024-11-01T16:52:25.8925328Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-11-01T16:52:25.8926163Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-11-01T16:52:25.8926660Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-11-01T16:52:25.8959147Z xfrm_user 57344 1 2024-11-01T16:52:25.8959623Z xfrm_algo 16384 1 xfrm_user 2024-11-01T16:52:25.8960099Z xt_addrtype 16384 2 2024-11-01T16:52:25.8960704Z nft_compat 20480 4 2024-11-01T16:52:25.8961194Z nf_tables 311296 57 nft_compat,nft_chain_nat 2024-11-01T16:52:25.8961908Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-11-01T16:52:25.8962559Z br_netfilter 36864 0 2024-11-01T16:52:25.8963043Z bridge 323584 1 br_netfilter 2024-11-01T16:52:25.8963621Z stp 16384 1 bridge 2024-11-01T16:52:25.8964115Z llc 16384 2 bridge,stp 2024-11-01T16:52:25.8964576Z overlay 167936 0 2024-11-01T16:52:25.8965004Z tls 114688 0 2024-11-01T16:52:25.8965433Z nls_ascii 16384 1 2024-11-01T16:52:25.8965872Z nls_cp437 20480 1 2024-11-01T16:52:25.8966301Z vfat 24576 1 2024-11-01T16:52:25.8966712Z sunrpc 692224 1 2024-11-01T16:52:25.8967118Z fat 86016 1 vfat 2024-11-01T16:52:25.8967547Z skx_edac 20480 0 2024-11-01T16:52:25.8967972Z skx_edac_common 28672 1 skx_edac 2024-11-01T16:52:25.8968441Z x86_pkg_temp_thermal 20480 0 2024-11-01T16:52:25.8968839Z kvm_intel 393216 0 2024-11-01T16:52:25.8969247Z kvm 1155072 1 kvm_intel 2024-11-01T16:52:25.8969707Z irqbypass 16384 1 kvm 2024-11-01T16:52:25.8970150Z ena 167936 0 2024-11-01T16:52:25.8970582Z ghash_clmulni_intel 16384 0 2024-11-01T16:52:25.8971003Z lpc_ich 28672 0 2024-11-01T16:52:25.8971425Z ioatdma 69632 0 2024-11-01T16:52:25.8971873Z mfd_core 20480 1 lpc_ich 2024-11-01T16:52:25.8972347Z dca 16384 1 ioatdma 2024-11-01T16:52:25.8972808Z button 24576 0 2024-11-01T16:52:25.8973239Z sch_fq_codel 20480 33 2024-11-01T16:52:25.8973655Z dm_mod 188416 0 2024-11-01T16:52:25.8974086Z fuse 163840 1 2024-11-01T16:52:25.8974506Z loop 36864 0 2024-11-01T16:52:25.8974930Z configfs 57344 1 2024-11-01T16:52:25.8975367Z dax 45056 1 dm_mod 2024-11-01T16:52:25.8975807Z dmi_sysfs 20480 0 2024-11-01T16:52:25.8976246Z crc32_pclmul 16384 0 2024-11-01T16:52:25.8976686Z crc32c_intel 24576 0 2024-11-01T16:52:25.8977110Z + modinfo nvidia 2024-11-01T16:52:25.8977932Z filename: /lib/modules/6.1.109-118.189.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-11-01T16:52:25.8978759Z alias: char-major-195-* 2024-11-01T16:52:25.8979208Z version: 550.54.15 2024-11-01T16:52:25.8979611Z supported: external 2024-11-01T16:52:25.8980007Z license: NVIDIA 2024-11-01T16:52:25.8980448Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-11-01T16:52:25.8980989Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-11-01T16:52:25.8981537Z srcversion: 833721318DA517F0C2FEC97 2024-11-01T16:52:25.8982083Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-11-01T16:52:25.8982673Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-11-01T16:52:25.8983255Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-11-01T16:52:25.8983820Z depends: i2c-core,drm 2024-11-01T16:52:25.8984213Z retpoline: Y 2024-11-01T16:52:25.8984564Z name: nvidia 2024-11-01T16:52:25.8985294Z vermagic: 6.1.109-118.189.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-11-01T16:52:25.8986148Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-11-01T16:52:25.8986915Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-11-01T16:52:25.8987599Z parm: NVreg_ResmanDebugLevel:int 2024-11-01T16:52:25.8988119Z parm: NVreg_RmLogonRC:int 2024-11-01T16:52:25.8988622Z parm: NVreg_ModifyDeviceFiles:int 2024-11-01T16:52:25.8989155Z parm: NVreg_DeviceFileUID:int 2024-11-01T16:52:25.8989731Z parm: NVreg_DeviceFileGID:int 2024-11-01T16:52:25.8990247Z parm: NVreg_DeviceFileMode:int 2024-11-01T16:52:25.8991059Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-11-01T16:52:25.8991680Z parm: NVreg_UsePageAttributeTable:int 2024-11-01T16:52:25.8992204Z parm: NVreg_EnablePCIeGen3:int 2024-11-01T16:52:25.8992671Z parm: NVreg_EnableMSI:int 2024-11-01T16:52:25.8993118Z parm: NVreg_TCEBypassMode:int 2024-11-01T16:52:25.8993642Z parm: NVreg_EnableStreamMemOPs:int 2024-11-01T16:52:25.8994204Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-11-01T16:52:25.8994840Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-11-01T16:52:25.8995449Z parm: NVreg_EnableS0ixPowerManagement:int 2024-11-01T16:52:25.8996103Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-11-01T16:52:25.8996758Z parm: NVreg_DynamicPowerManagement:int 2024-11-01T16:52:25.8997409Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-11-01T16:52:25.8998053Z parm: NVreg_EnableGpuFirmware:int 2024-11-01T16:52:25.8998580Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-11-01T16:52:25.8999158Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-11-01T16:52:25.8999753Z parm: NVreg_EnableUserNUMAManagement:int 2024-11-01T16:52:25.9000279Z parm: NVreg_MemoryPoolSize:int 2024-11-01T16:52:25.9000778Z parm: NVreg_KMallocHeapMaxSize:int 2024-11-01T16:52:25.9001303Z parm: NVreg_VMallocHeapMaxSize:int 2024-11-01T16:52:25.9001811Z parm: NVreg_IgnoreMMIOCheck:int 2024-11-01T16:52:25.9002465Z parm: NVreg_NvLinkDisable:int 2024-11-01T16:52:25.9003045Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-11-01T16:52:25.9003789Z parm: NVreg_RegisterPCIDriver:int 2024-11-01T16:52:25.9004350Z parm: NVreg_EnableResizableBar:int 2024-11-01T16:52:25.9004922Z parm: NVreg_EnableDbgBreakpoint:int 2024-11-01T16:52:25.9005508Z parm: NVreg_EnableNonblockingOpen:int 2024-11-01T16:52:25.9006077Z parm: NVreg_RegistryDwords:charp 2024-11-01T16:52:25.9006642Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-11-01T16:52:25.9007203Z parm: NVreg_RmMsg:charp 2024-11-01T16:52:25.9007689Z parm: NVreg_GpuBlacklist:charp 2024-11-01T16:52:25.9008240Z parm: NVreg_TemporaryFilePath:charp 2024-11-01T16:52:25.9008788Z parm: NVreg_ExcludedGpus:charp 2024-11-01T16:52:25.9009301Z parm: NVreg_DmaRemapPeerMmio:int 2024-11-01T16:52:25.9009916Z parm: NVreg_RmNvlinkBandwidth:charp 2024-11-01T16:52:25.9010608Z parm: NVreg_ImexChannelCount:int 2024-11-01T16:52:25.9011213Z parm: rm_firmware_active:charp 2024-11-01T16:52:25.9011691Z + HAS_NVIDIA_DRIVER=0 2024-11-01T16:52:25.9012132Z ++ command -v nvidia-smi 2024-11-01T16:52:25.9012573Z + '[' -x /usr/bin/nvidia-smi ']' 2024-11-01T16:52:25.9013003Z + set +e 2024-11-01T16:52:25.9013606Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-11-01T16:52:25.9014316Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-11-01T16:52:25.9014773Z + NVIDIA_SMI_STATUS=0 2024-11-01T16:52:25.9015154Z + '[' 0 -ne 0 ']' 2024-11-01T16:52:25.9015564Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-11-01T16:52:25.9016007Z + HAS_NVIDIA_DRIVER=1 2024-11-01T16:52:25.9016858Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-11-01T16:52:25.9017698Z + set -e 2024-11-01T16:52:25.9018041Z + '[' 1 -eq 0 ']' 2024-11-01T16:52:25.9018685Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-11-01T16:52:25.9019488Z + post_install_nvidia_driver_common 2024-11-01T16:52:25.9019953Z + sudo modprobe nvidia 2024-11-01T16:52:26.0280119Z + echo 'After installing NVIDIA driver' 2024-11-01T16:52:26.0280662Z + lspci 2024-11-01T16:52:26.0281024Z After installing NVIDIA driver 2024-11-01T16:52:26.0524208Z 00:00.0 Host bridge: Intel Corporation Sky Lake-E DMI3 Registers (rev 07) 2024-11-01T16:52:26.0525500Z 00:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0526572Z 00:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0527666Z 00:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0528766Z 00:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0529969Z 00:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0531060Z 00:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0532136Z 00:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0533208Z 00:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0534391Z 00:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 07) 2024-11-01T16:52:26.0535511Z 00:05.2 System peripheral: Intel Corporation Sky Lake-E RAS (rev 07) 2024-11-01T16:52:26.0536567Z 00:05.4 PIC: Intel Corporation Sky Lake-E IOAPIC (rev 07) 2024-11-01T16:52:26.0537489Z 00:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:26.0538595Z 00:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:26.0539697Z 00:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:26.0540768Z 00:11.0 Unassigned class [ff00]: Intel Corporation C620 Series Chipset Family MROM 0 (rev 09) 2024-11-01T16:52:26.0541914Z 00:11.1 Unassigned class [ff00]: Intel Corporation C620 Series Chipset Family MROM 1 (rev 09) 2024-11-01T16:52:26.0543145Z 00:11.5 SATA controller: Intel Corporation C620 Series Chipset Family SSATA Controller [AHCI mode] (rev 09) 2024-11-01T16:52:26.0544428Z 00:14.0 USB controller: Intel Corporation C620 Series Chipset Family USB 3.0 xHCI Controller (rev 09) 2024-11-01T16:52:26.0545712Z 00:14.2 Signal processing controller: Intel Corporation C620 Series Chipset Family Thermal Subsystem (rev 09) 2024-11-01T16:52:26.0547050Z 00:16.0 Communication controller: Intel Corporation C620 Series Chipset Family MEI Controller #1 (rev 09) 2024-11-01T16:52:26.0548354Z 00:16.4 Communication controller: Intel Corporation C620 Series Chipset Family MEI Controller #3 (rev 09) 2024-11-01T16:52:26.0549675Z 00:17.0 SATA controller: Intel Corporation C620 Series Chipset Family SATA Controller [AHCI mode] (rev 09) 2024-11-01T16:52:26.0550866Z 00:1f.0 ISA bridge: Intel Corporation C621 Series Chipset LPC/eSPI Controller (rev 09) 2024-11-01T16:52:26.0552090Z 00:1f.2 Memory controller: Intel Corporation C620 Series Chipset Family Power Management Controller (rev 09) 2024-11-01T16:52:26.0553235Z 00:1f.4 SMBus: Intel Corporation C620 Series Chipset Family SMBus (rev 09) 2024-11-01T16:52:26.0554297Z 00:1f.5 Serial bus controller: Intel Corporation C620 Series Chipset Family SPI Controller (rev 09) 2024-11-01T16:52:26.0555520Z 17:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:26.0556625Z 17:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:26.0557640Z 17:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:26.0558748Z 17:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:26.0559941Z 17:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:26.0561000Z 17:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0562056Z 17:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0563107Z 17:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0564239Z 17:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0565290Z 17:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0566336Z 17:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0567383Z 17:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0568409Z 17:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0569450Z 17:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0571885Z 17:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0573094Z 17:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0574137Z 17:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0575183Z 17:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0576210Z 17:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0577249Z 17:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0578292Z 17:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0579343Z 17:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0580394Z 17:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0581447Z 17:0a.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0582472Z 17:0a.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0583516Z 17:0a.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0584589Z 17:0a.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0585644Z 17:0a.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0586687Z 17:0a.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0587737Z 17:0b.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0588853Z 17:0b.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0589872Z 17:0b.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0591080Z 17:0b.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0592129Z 17:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0593162Z 17:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0594208Z 17:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0595242Z 17:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0596263Z 17:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0597302Z 17:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0598345Z 17:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0599388Z 17:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0600426Z 17:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0601457Z 17:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0602475Z 17:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0603512Z 17:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0604553Z 17:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0605690Z 17:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0606738Z 17:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0607788Z 17:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0608809Z 17:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0609936Z 17:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0611051Z 17:10.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0612102Z 17:10.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0613147Z 17:10.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0614200Z 17:10.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0615240Z 17:10.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0616267Z 17:10.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0617305Z 17:11.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0618346Z 17:11.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0619382Z 17:11.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0620432Z 17:11.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0621485Z 17:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0622605Z 17:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0623566Z 17:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0624523Z 17:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0625491Z 17:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0626453Z 17:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0627412Z 17:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0628349Z 17:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0629311Z 17:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0630274Z 17:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0631233Z 17:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0632069Z 18:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:26.0632841Z 19:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:26.0633763Z 34:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:26.0634744Z 34:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:26.0635676Z 34:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:26.0636701Z 34:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:26.0637778Z 34:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:26.0638877Z 34:08.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0640037Z 34:09.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0641170Z 34:0a.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0642324Z 34:0a.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0643659Z 34:0a.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0645147Z 34:0a.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0646405Z 34:0a.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0647555Z 34:0a.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07) 2024-11-01T16:52:26.0648601Z 34:0a.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07) 2024-11-01T16:52:26.0649716Z 34:0a.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07) 2024-11-01T16:52:26.0650788Z 34:0b.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07) 2024-11-01T16:52:26.0651843Z 34:0b.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07) 2024-11-01T16:52:26.0652895Z 34:0b.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07) 2024-11-01T16:52:26.0653963Z 34:0b.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07) 2024-11-01T16:52:26.0655235Z 34:0c.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0656430Z 34:0c.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0657644Z 34:0c.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0658855Z 34:0c.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0660076Z 34:0c.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0661183Z 34:0c.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07) 2024-11-01T16:52:26.0662207Z 34:0c.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07) 2024-11-01T16:52:26.0663237Z 34:0c.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07) 2024-11-01T16:52:26.0664244Z 34:0d.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07) 2024-11-01T16:52:26.0665272Z 34:0d.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07) 2024-11-01T16:52:26.0666287Z 34:0d.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07) 2024-11-01T16:52:26.0667309Z 34:0d.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07) 2024-11-01T16:52:26.0668191Z 35:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:26.0669012Z 36:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:26.0669972Z 7a:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:26.0671032Z 7a:01.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port B (rev 07) 2024-11-01T16:52:26.0672083Z 7a:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:26.0673061Z 7a:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:26.0674138Z 7a:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:26.0675292Z 7a:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:26.0676438Z 7a:0e.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:26.0677445Z 7a:0e.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:26.0678442Z 7a:0f.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:26.0679448Z 7a:0f.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:26.0680434Z 7a:10.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:26.0681428Z 7a:10.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:26.0682464Z 7a:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0683553Z 7a:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0684759Z 7a:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0685780Z 7a:12.4 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0686810Z 7a:12.5 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0687830Z 7a:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:26.0689006Z 7a:15.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:26.0690105Z 7a:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:26.0691372Z 7a:16.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:26.0692469Z 7a:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:26.0693576Z 7a:16.5 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:26.0694678Z 7a:17.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:26.0695774Z 7a:17.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:26.0696577Z 7b:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0697195Z 7c:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0697805Z 7c:01.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0698408Z 7c:02.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0699015Z 7c:03.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0699611Z 7c:04.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0700188Z 7c:05.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0700898Z 7c:06.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0701491Z 7c:07.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0702188Z 7c:08.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0702745Z 7c:09.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0703274Z 7c:0a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0703834Z 7c:0b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0704381Z 7c:0c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0704933Z 7c:0d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0705491Z 7c:0e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0706040Z 7c:0f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0706577Z 7c:10.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0707134Z 7c:11.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0707686Z 7c:12.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0708235Z 7c:13.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0708790Z 7c:14.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0709324Z 7c:15.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0710103Z 7c:16.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0710701Z 7c:17.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0711302Z 7c:18.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0711905Z 7c:19.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0712502Z 7c:1a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0713079Z 7c:1b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0713687Z 7c:1c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0714302Z 7c:1d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0714905Z 7c:1e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0715511Z 7c:1f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0716086Z 9d:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0716688Z 9e:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0717289Z 9e:01.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0717974Z 9e:02.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0718578Z 9e:03.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0719176Z 9e:04.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0719747Z 9e:05.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0720348Z 9e:06.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0720948Z 9e:07.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0721543Z 9e:08.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0722202Z 9e:09.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0722774Z 9e:0a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0723376Z 9e:0b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0724098Z 9e:0c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0724780Z 9e:0d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0725320Z 9e:0e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0725865Z 9e:0f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0726380Z 9e:10.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0727106Z 9e:11.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0727688Z 9e:12.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0728437Z 9e:13.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0729023Z 9e:14.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0729585Z 9e:15.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0730162Z 9e:16.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0730736Z 9e:17.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0731323Z 9e:18.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0731907Z 9e:19.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0732476Z 9e:1a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0733063Z 9e:1b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0733647Z 9e:1c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0734233Z 9e:1d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0734817Z 9e:1e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0735408Z 9e:1f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0736018Z 9f:00.0 Serial controller: Amazon.com, Inc. Device 8250 2024-11-01T16:52:26.0736873Z a0:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device 0061 2024-11-01T16:52:26.0737797Z bf:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device cd00 2024-11-01T16:52:26.0738828Z bf:00.1 Non-Volatile memory controller: Amazon.com, Inc. Device cd00 2024-11-01T16:52:26.0739919Z c0:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0740912Z c0:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0741879Z c0:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0742877Z c0:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0744050Z c0:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0745087Z c0:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0746120Z c0:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0747154Z c0:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07) 2024-11-01T16:52:26.0748294Z c0:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 07) 2024-11-01T16:52:26.0749368Z c0:05.2 System peripheral: Intel Corporation Sky Lake-E RAS (rev 07) 2024-11-01T16:52:26.0750197Z c0:05.4 PIC: Intel Corporation Sky Lake-E IOAPIC (rev 07) 2024-11-01T16:52:26.0751114Z c0:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:26.0752165Z c0:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:26.0753246Z c0:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07) 2024-11-01T16:52:26.0754280Z c2:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:26.0755232Z c2:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:26.0756280Z c2:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:26.0757395Z c2:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:26.0758485Z c2:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0759484Z c2:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0760479Z c2:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0761458Z c2:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0762451Z c2:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0763443Z c2:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0764433Z c2:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0765416Z c2:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0766402Z c2:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0767379Z c2:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0768368Z c2:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0769358Z c2:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0770489Z c2:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0771476Z c2:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0772470Z c2:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0773443Z c2:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0774431Z c2:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0775585Z c2:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0776572Z c2:0a.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0777676Z c2:0a.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0778609Z c2:0a.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0779615Z c2:0a.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0780580Z c2:0a.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0781513Z c2:0a.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0782446Z c2:0b.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0783356Z c2:0b.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0784280Z c2:0b.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0785214Z c2:0b.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0786162Z c2:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0787093Z c2:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0788029Z c2:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0788965Z c2:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0789887Z c2:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0790869Z c2:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0791795Z c2:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0792722Z c2:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0793653Z c2:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0794578Z c2:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0795586Z c2:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0796511Z c2:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0797441Z c2:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0798373Z c2:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0799307Z c2:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0800237Z c2:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0801157Z c2:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0802089Z c2:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0803202Z c2:10.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0804361Z c2:10.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0805374Z c2:10.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0806385Z c2:10.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0807380Z c2:10.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0808384Z c2:10.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0809403Z c2:11.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0810468Z c2:11.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0811488Z c2:11.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0812505Z c2:11.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0813523Z c2:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0814531Z c2:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0815549Z c2:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0816562Z c2:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07) 2024-11-01T16:52:26.0817580Z c2:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0818596Z c2:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0819612Z c2:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0820615Z c2:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0821633Z c2:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0822832Z c2:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0823786Z c2:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07) 2024-11-01T16:52:26.0824518Z c3:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0825089Z c4:00.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0825619Z c4:01.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0826164Z c4:02.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0826693Z c4:03.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0827300Z c4:04.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0827824Z c4:05.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0828350Z c4:06.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0828860Z c4:07.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0829389Z c4:08.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0829933Z c4:09.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0830490Z c4:0a.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0831101Z c4:0b.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0831631Z c4:0c.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0832178Z c4:0d.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0832735Z c4:0e.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0833287Z c4:0f.0 PCI bridge: Amazon.com, Inc. Device bec2 2024-11-01T16:52:26.0833988Z c6:00.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-11-01T16:52:26.0834973Z e6:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:26.0835961Z e6:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:26.0836892Z e6:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:26.0837913Z e6:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:26.0839002Z e6:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:26.0840275Z e6:08.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0841505Z e6:09.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0842688Z e6:0a.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0843903Z e6:0a.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0845119Z e6:0a.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0846324Z e6:0a.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0847533Z e6:0a.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0848626Z e6:0a.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07) 2024-11-01T16:52:26.0849651Z e6:0a.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07) 2024-11-01T16:52:26.0850662Z e6:0a.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07) 2024-11-01T16:52:26.0851677Z e6:0b.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07) 2024-11-01T16:52:26.0852694Z e6:0b.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07) 2024-11-01T16:52:26.0853710Z e6:0b.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07) 2024-11-01T16:52:26.0854746Z e6:0b.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07) 2024-11-01T16:52:26.0855867Z e6:0c.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0857061Z e6:0c.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0858270Z e6:0c.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0859494Z e6:0c.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0860703Z e6:0c.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07) 2024-11-01T16:52:26.0861907Z e6:0c.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07) 2024-11-01T16:52:26.0862875Z e6:0c.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07) 2024-11-01T16:52:26.0864352Z e6:0c.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07) 2024-11-01T16:52:26.0865398Z e6:0d.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07) 2024-11-01T16:52:26.0866433Z e6:0d.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07) 2024-11-01T16:52:26.0867470Z e6:0d.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07) 2024-11-01T16:52:26.0868516Z e6:0d.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07) 2024-11-01T16:52:26.0869475Z e7:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:26.0870308Z e8:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:26.0871295Z f3:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07) 2024-11-01T16:52:26.0872391Z f3:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07) 2024-11-01T16:52:26.0873403Z f3:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07) 2024-11-01T16:52:26.0874500Z f3:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07) 2024-11-01T16:52:26.0875758Z f3:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07) 2024-11-01T16:52:26.0876945Z f3:0e.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:26.0877956Z f3:0e.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:26.0878983Z f3:0f.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:26.0880012Z f3:0f.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:26.0881028Z f3:10.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07) 2024-11-01T16:52:26.0882039Z f3:10.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07) 2024-11-01T16:52:26.0883104Z f3:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0884229Z f3:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0885306Z f3:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0886402Z f3:12.4 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0887499Z f3:12.5 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07) 2024-11-01T16:52:26.0888581Z f3:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:26.0889662Z f3:15.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:26.0890744Z f3:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:26.0891800Z f3:16.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:26.0892872Z f3:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:26.0893938Z f3:16.5 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:26.0895023Z f3:17.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07) 2024-11-01T16:52:26.0896114Z f3:17.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07) 2024-11-01T16:52:26.0897052Z f4:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:26.0897869Z f5:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) 2024-11-01T16:52:26.0898505Z + lsmod 2024-11-01T16:52:26.0898843Z Module Size Used by 2024-11-01T16:52:26.0899298Z veth 36864 0 2024-11-01T16:52:26.0899724Z nvidia_modeset 1351680 0 2024-11-01T16:52:26.0900186Z video 65536 1 nvidia_modeset 2024-11-01T16:52:26.0900657Z wmi 36864 1 video 2024-11-01T16:52:26.0901098Z nvidia_uvm 4706304 0 2024-11-01T16:52:26.0901609Z nvidia 54071296 35 nvidia_uvm,nvidia_modeset 2024-11-01T16:52:26.0902171Z drm 602112 1 nvidia 2024-11-01T16:52:26.0902738Z drm_panel_orientation_quirks 32768 1 drm 2024-11-01T16:52:26.0903330Z backlight 24576 3 video,drm,nvidia_modeset 2024-11-01T16:52:26.0903923Z i2c_core 106496 2 nvidia,drm 2024-11-01T16:52:26.0904520Z xt_conntrack 16384 1 2024-11-01T16:52:26.0905059Z nft_chain_nat 16384 3 2024-11-01T16:52:26.0905460Z xt_MASQUERADE 20480 1 2024-11-01T16:52:26.0905917Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-11-01T16:52:26.0906673Z nf_conntrack_netlink 57344 0 2024-11-01T16:52:26.0907333Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-11-01T16:52:26.0908081Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-11-01T16:52:26.0908605Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-11-01T16:52:26.0909097Z xfrm_user 57344 1 2024-11-01T16:52:26.0909574Z xfrm_algo 16384 1 xfrm_user 2024-11-01T16:52:26.0910213Z xt_addrtype 16384 2 2024-11-01T16:52:26.0910745Z nft_compat 20480 4 2024-11-01T16:52:26.0911253Z nf_tables 311296 57 nft_compat,nft_chain_nat 2024-11-01T16:52:26.0911946Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-11-01T16:52:26.0912562Z br_netfilter 36864 0 2024-11-01T16:52:26.0913029Z bridge 323584 1 br_netfilter 2024-11-01T16:52:26.0913526Z stp 16384 1 bridge 2024-11-01T16:52:26.0913990Z llc 16384 2 bridge,stp 2024-11-01T16:52:26.0914475Z overlay 167936 0 2024-11-01T16:52:26.0914881Z tls 114688 0 2024-11-01T16:52:26.0915277Z nls_ascii 16384 1 2024-11-01T16:52:26.0915704Z nls_cp437 20480 1 2024-11-01T16:52:26.0916118Z vfat 24576 1 2024-11-01T16:52:26.0916520Z sunrpc 692224 1 2024-11-01T16:52:26.0916920Z fat 86016 1 vfat 2024-11-01T16:52:26.0917336Z skx_edac 20480 0 2024-11-01T16:52:26.0917762Z skx_edac_common 28672 1 skx_edac 2024-11-01T16:52:26.0918225Z x86_pkg_temp_thermal 20480 0 2024-11-01T16:52:26.0918645Z kvm_intel 393216 0 2024-11-01T16:52:26.0919089Z kvm 1155072 1 kvm_intel 2024-11-01T16:52:26.0919546Z irqbypass 16384 1 kvm 2024-11-01T16:52:26.0919996Z ena 167936 0 2024-11-01T16:52:26.0920414Z ghash_clmulni_intel 16384 0 2024-11-01T16:52:26.0920852Z lpc_ich 28672 0 2024-11-01T16:52:26.0921265Z ioatdma 69632 0 2024-11-01T16:52:26.0921681Z mfd_core 20480 1 lpc_ich 2024-11-01T16:52:26.0922154Z dca 16384 1 ioatdma 2024-11-01T16:52:26.0922609Z button 24576 0 2024-11-01T16:52:26.0923035Z sch_fq_codel 20480 33 2024-11-01T16:52:26.0923558Z dm_mod 188416 0 2024-11-01T16:52:26.0923934Z fuse 163840 1 2024-11-01T16:52:26.0924343Z loop 36864 0 2024-11-01T16:52:26.0924762Z configfs 57344 1 2024-11-01T16:52:26.0925181Z dax 45056 1 dm_mod 2024-11-01T16:52:26.0925628Z dmi_sysfs 20480 0 2024-11-01T16:52:26.0926015Z crc32_pclmul 16384 0 2024-11-01T16:52:26.0926424Z crc32c_intel 24576 0 2024-11-01T16:52:26.0926838Z + modinfo nvidia 2024-11-01T16:52:26.0927684Z filename: /lib/modules/6.1.109-118.189.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-11-01T16:52:26.0928447Z alias: char-major-195-* 2024-11-01T16:52:26.0928856Z version: 550.54.15 2024-11-01T16:52:26.0929197Z supported: external 2024-11-01T16:52:26.0929554Z license: NVIDIA 2024-11-01T16:52:26.0929943Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-11-01T16:52:26.0930451Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-11-01T16:52:26.0930939Z srcversion: 833721318DA517F0C2FEC97 2024-11-01T16:52:26.0931411Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-11-01T16:52:26.0932042Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-11-01T16:52:26.0932565Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-11-01T16:52:26.0933096Z depends: i2c-core,drm 2024-11-01T16:52:26.0933481Z retpoline: Y 2024-11-01T16:52:26.0933768Z name: nvidia 2024-11-01T16:52:26.0934419Z vermagic: 6.1.109-118.189.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-11-01T16:52:26.0935183Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-11-01T16:52:26.0935960Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-11-01T16:52:26.0936592Z parm: NVreg_ResmanDebugLevel:int 2024-11-01T16:52:26.0937065Z parm: NVreg_RmLogonRC:int 2024-11-01T16:52:26.0937500Z parm: NVreg_ModifyDeviceFiles:int 2024-11-01T16:52:26.0937978Z parm: NVreg_DeviceFileUID:int 2024-11-01T16:52:26.0938449Z parm: NVreg_DeviceFileGID:int 2024-11-01T16:52:26.0938914Z parm: NVreg_DeviceFileMode:int 2024-11-01T16:52:26.0939485Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-11-01T16:52:26.0940071Z parm: NVreg_UsePageAttributeTable:int 2024-11-01T16:52:26.0940595Z parm: NVreg_EnablePCIeGen3:int 2024-11-01T16:52:26.0941065Z parm: NVreg_EnableMSI:int 2024-11-01T16:52:26.0941514Z parm: NVreg_TCEBypassMode:int 2024-11-01T16:52:26.0941998Z parm: NVreg_EnableStreamMemOPs:int 2024-11-01T16:52:26.0942577Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-11-01T16:52:26.0943192Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-11-01T16:52:26.0943800Z parm: NVreg_EnableS0ixPowerManagement:int 2024-11-01T16:52:26.0944459Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-11-01T16:52:26.0945099Z parm: NVreg_DynamicPowerManagement:int 2024-11-01T16:52:26.0945739Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-11-01T16:52:26.0946347Z parm: NVreg_EnableGpuFirmware:int 2024-11-01T16:52:26.0946853Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-11-01T16:52:26.0947401Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-11-01T16:52:26.0947964Z parm: NVreg_EnableUserNUMAManagement:int 2024-11-01T16:52:26.0948490Z parm: NVreg_MemoryPoolSize:int 2024-11-01T16:52:26.0948956Z parm: NVreg_KMallocHeapMaxSize:int 2024-11-01T16:52:26.0949118Z parm: NVreg_VMallocHeapMaxSize:int 2024-11-01T16:52:26.0949265Z parm: NVreg_IgnoreMMIOCheck:int 2024-11-01T16:52:26.0949423Z parm: NVreg_NvLinkDisable:int 2024-11-01T16:52:26.0949643Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-11-01T16:52:26.0949832Z parm: NVreg_RegisterPCIDriver:int 2024-11-01T16:52:26.0950003Z parm: NVreg_EnableResizableBar:int 2024-11-01T16:52:26.0950192Z parm: NVreg_EnableDbgBreakpoint:int 2024-11-01T16:52:26.0950378Z parm: NVreg_EnableNonblockingOpen:int 2024-11-01T16:52:26.0950540Z parm: NVreg_RegistryDwords:charp 2024-11-01T16:52:26.0950762Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-11-01T16:52:26.0950894Z parm: NVreg_RmMsg:charp 2024-11-01T16:52:26.0951070Z parm: NVreg_GpuBlacklist:charp 2024-11-01T16:52:26.0951246Z parm: NVreg_TemporaryFilePath:charp 2024-11-01T16:52:26.0951422Z parm: NVreg_ExcludedGpus:charp 2024-11-01T16:52:26.0951582Z parm: NVreg_DmaRemapPeerMmio:int 2024-11-01T16:52:26.0951755Z parm: NVreg_RmNvlinkBandwidth:charp 2024-11-01T16:52:26.0951940Z parm: NVreg_ImexChannelCount:int 2024-11-01T16:52:26.0952090Z parm: rm_firmware_active:charp 2024-11-01T16:52:26.0952222Z + set +e 2024-11-01T16:52:26.0952369Z + nvidia-smi 2024-11-01T16:52:26.1390012Z Fri Nov 1 16:52:26 2024 2024-11-01T16:52:26.1390615Z +-----------------------------------------------------------------------------------------+ 2024-11-01T16:52:26.1391273Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-11-01T16:52:26.1391683Z |-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:52:26.1392161Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-11-01T16:52:26.1392759Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-11-01T16:52:26.1392989Z | | | MIG M. | 2024-11-01T16:52:26.1394571Z |=========================================+========================+======================| 2024-11-01T16:52:26.1574403Z | 0 Tesla T4 On | 00000000:18:00.0 Off | 0 | 2024-11-01T16:52:26.1575359Z | N/A 31C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:52:26.1575612Z | | | N/A | 2024-11-01T16:52:26.1576162Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:52:26.1720490Z | 1 Tesla T4 On | 00000000:19:00.0 Off | 0 | 2024-11-01T16:52:26.1720854Z | N/A 31C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:52:26.1721075Z | | | N/A | 2024-11-01T16:52:26.1721511Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:52:26.1861203Z | 2 Tesla T4 On | 00000000:35:00.0 Off | 0 | 2024-11-01T16:52:26.1861702Z | N/A 30C P8 10W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:52:26.1861921Z | | | N/A | 2024-11-01T16:52:26.1862360Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:52:26.2008922Z | 3 Tesla T4 On | 00000000:36:00.0 Off | 0 | 2024-11-01T16:52:26.2010187Z | N/A 31C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:52:26.2010762Z | | | N/A | 2024-11-01T16:52:26.2011672Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:52:26.2152548Z | 4 Tesla T4 On | 00000000:E7:00.0 Off | 0 | 2024-11-01T16:52:26.2153406Z | N/A 31C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:52:26.2153999Z | | | N/A | 2024-11-01T16:52:26.2154652Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:52:26.2296776Z | 5 Tesla T4 On | 00000000:E8:00.0 Off | 0 | 2024-11-01T16:52:26.2297211Z | N/A 31C P8 10W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:52:26.2297451Z | | | N/A | 2024-11-01T16:52:26.2297866Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:52:26.2442554Z | 6 Tesla T4 On | 00000000:F4:00.0 Off | 0 | 2024-11-01T16:52:26.2443169Z | N/A 31C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:52:26.2443409Z | | | N/A | 2024-11-01T16:52:26.2443822Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:52:26.2587419Z | 7 Tesla T4 On | 00000000:F5:00.0 Off | 0 | 2024-11-01T16:52:26.2588303Z | N/A 32C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:52:26.2588879Z | | | N/A | 2024-11-01T16:52:26.2589932Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:52:26.2621895Z 2024-11-01T16:52:26.2624431Z +-----------------------------------------------------------------------------------------+ 2024-11-01T16:52:26.2624710Z | Processes: | 2024-11-01T16:52:26.2625090Z | GPU GI CI PID Type Process name GPU Memory | 2024-11-01T16:52:26.2625336Z | ID ID Usage | 2024-11-01T16:52:26.2625561Z |=========================================================================================| 2024-11-01T16:52:26.2731697Z | No running processes found | 2024-11-01T16:52:26.2732381Z +-----------------------------------------------------------------------------------------+ 2024-11-01T16:52:27.4054567Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-11-01T16:52:27.4246395Z Tesla T4 2024-11-01T16:52:27.4310817Z + NVIDIA_SMI_STATUS=0 2024-11-01T16:52:27.4312004Z + '[' 0 -eq 0 ']' 2024-11-01T16:52:27.4313132Z + echo 'INFO: Ignoring allowed status 0' 2024-11-01T16:52:27.4314173Z + set -e 2024-11-01T16:52:27.4314475Z INFO: Ignoring allowed status 0 2024-11-01T16:52:27.4319210Z == Installing nvidia container toolkit for amzn2023 == 2024-11-01T16:52:27.4323300Z + sudo yum install -y yum-utils 2024-11-01T16:52:27.8498556Z Last metadata expiration check: 0:20:49 ago on Fri Nov 1 16:31:38 2024. 2024-11-01T16:52:27.8742532Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-11-01T16:52:27.9121515Z Dependencies resolved. 2024-11-01T16:52:27.9299241Z Nothing to do. 2024-11-01T16:52:27.9300107Z Complete! 2024-11-01T16:52:28.0503724Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-11-01T16:52:28.0504986Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-11-01T16:52:28.0506637Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-11-01T16:52:28.2981070Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-11-01T16:52:28.3668091Z + sudo yum install -y nvidia-docker2 2024-11-01T16:52:28.8712939Z nvidia-container-toolkit 11 kB/s | 833 B 00:00 2024-11-01T16:52:28.8960469Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-11-01T16:52:28.9326635Z Dependencies resolved. 2024-11-01T16:52:28.9501424Z Nothing to do. 2024-11-01T16:52:28.9502431Z Complete! 2024-11-01T16:52:29.0804525Z + sudo systemctl restart docker 2024-11-01T16:54:11.9660871Z nvidia-persistenced failed to initialize. Check syslog for more details. 2024-11-01T16:54:12.0507405Z Fri Nov 1 16:54:12 2024 2024-11-01T16:54:12.0508275Z +-----------------------------------------------------------------------------------------+ 2024-11-01T16:54:12.0509250Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-11-01T16:54:12.0510286Z |-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:54:12.0511230Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-11-01T16:54:12.0512286Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-11-01T16:54:12.0513060Z | | | MIG M. | 2024-11-01T16:54:12.0513950Z |=========================================+========================+======================| 2024-11-01T16:54:12.0691308Z | 0 Tesla T4 On | 00000000:18:00.0 Off | 0 | 2024-11-01T16:54:12.0692255Z | N/A 31C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:54:12.0692991Z | | | N/A | 2024-11-01T16:54:12.0694073Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:54:12.0837824Z | 1 Tesla T4 On | 00000000:19:00.0 Off | 0 | 2024-11-01T16:54:12.0838755Z | N/A 31C P8 10W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:54:12.0839482Z | | | N/A | 2024-11-01T16:54:12.0840323Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:54:12.0976964Z | 2 Tesla T4 On | 00000000:35:00.0 Off | 0 | 2024-11-01T16:54:12.0977876Z | N/A 30C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:54:12.0978647Z | | | N/A | 2024-11-01T16:54:12.0979513Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:54:12.1120044Z | 3 Tesla T4 On | 00000000:36:00.0 Off | 0 | 2024-11-01T16:54:12.1121775Z | N/A 31C P8 10W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:54:12.1122667Z | | | N/A | 2024-11-01T16:54:12.1123521Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:54:12.1265323Z | 4 Tesla T4 On | 00000000:E7:00.0 Off | 0 | 2024-11-01T16:54:12.1266225Z | N/A 31C P8 10W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:54:12.1267001Z | | | N/A | 2024-11-01T16:54:12.1267871Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:54:12.1412076Z | 5 Tesla T4 On | 00000000:E8:00.0 Off | 0 | 2024-11-01T16:54:12.1413160Z | N/A 31C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:54:12.1413953Z | | | N/A | 2024-11-01T16:54:12.1414840Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:54:12.1554290Z | 6 Tesla T4 On | 00000000:F4:00.0 Off | 0 | 2024-11-01T16:54:12.1555260Z | N/A 31C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:54:12.1556029Z | | | N/A | 2024-11-01T16:54:12.1556899Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:54:12.1710765Z | 7 Tesla T4 On | 00000000:F5:00.0 Off | 0 | 2024-11-01T16:54:12.1711652Z | N/A 32C P8 9W / 70W | 0MiB / 15360MiB | 0% Default | 2024-11-01T16:54:12.1712413Z | | | N/A | 2024-11-01T16:54:12.1713281Z +-----------------------------------------+------------------------+----------------------+ 2024-11-01T16:54:12.1787288Z 2024-11-01T16:54:12.1788521Z +-----------------------------------------------------------------------------------------+ 2024-11-01T16:54:12.1789320Z | Processes: | 2024-11-01T16:54:12.1790178Z | GPU GI CI PID Type Process name GPU Memory | 2024-11-01T16:54:12.1790976Z | ID ID Usage | 2024-11-01T16:54:12.1791726Z |=========================================================================================| 2024-11-01T16:54:12.1894354Z | No running processes found | 2024-11-01T16:54:12.1895385Z +-----------------------------------------------------------------------------------------+ 2024-11-01T16:54:13.4903745Z Command completed after 1 attempt(s). 2024-11-01T16:54:13.5010440Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-11-01T16:54:13.5011493Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-11-01T16:54:13.5012312Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-11-01T16:54:13.5013088Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-11-01T16:54:13.5021807Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:13.5022378Z env: 2024-11-01T16:54:13.5022703Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:13.5023236Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:13.5023915Z ##[endgroup] 2024-11-01T16:54:13.7435342Z Defaulting to user installation because normal site-packages is not writeable 2024-11-01T16:54:13.7591729Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-11-01T16:54:13.7596154Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-11-01T16:54:13.8798814Z Prepare all required actions 2024-11-01T16:54:13.8799352Z Getting action download info 2024-11-01T16:54:13.9933633Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-11-01T16:54:14.1675476Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-11-01T16:54:14.2862674Z ##[group]Run ./.github/actions/download-build-artifacts 2024-11-01T16:54:14.2863180Z with: 2024-11-01T16:54:14.2863514Z name: linux-focal-cuda12.1-py3.10-gcc9-sm75 2024-11-01T16:54:14.2864164Z s3-bucket: gha-artifacts 2024-11-01T16:54:14.2864516Z env: 2024-11-01T16:54:14.2864797Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:14.2865271Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:14.2865792Z ##[endgroup] 2024-11-01T16:54:14.2906150Z ##[group]Run seemethere/download-artifact-s3@v4 2024-11-01T16:54:14.2906656Z with: 2024-11-01T16:54:14.2907070Z name: linux-focal-cuda12.1-py3.10-gcc9-sm75 2024-11-01T16:54:14.2907591Z s3-bucket: gha-artifacts 2024-11-01T16:54:14.2907964Z region: us-east-1 2024-11-01T16:54:14.2908280Z env: 2024-11-01T16:54:14.2908584Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:14.2909059Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:14.2909853Z ##[endgroup] 2024-11-01T16:54:14.7470899Z (node:83471) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-11-01T16:54:14.7473071Z 2024-11-01T16:54:14.7473860Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-11-01T16:54:14.7476029Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-11-01T16:54:14.7478694Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-11-01T16:54:14.8128623Z Found 1 objects with prefix pytorch/pytorch/11632514903/linux-focal-cuda12.1-py3.10-gcc9-sm75/ 2024-11-01T16:54:14.8129899Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-11-01T16:54:34.3309868Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-11-01T16:54:34.3316378Z Artifact download has finished successfully 2024-11-01T16:54:34.3529507Z ##[group]Run unzip -o artifacts.zip 2024-11-01T16:54:34.3529952Z unzip -o artifacts.zip 2024-11-01T16:54:34.3535532Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:34.3536046Z env: 2024-11-01T16:54:34.3536327Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:34.3536998Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:34.3537506Z ##[endgroup] 2024-11-01T16:54:34.3575710Z Archive: artifacts.zip 2024-11-01T16:54:34.3576797Z creating: dist/ 2024-11-01T16:54:36.2246885Z inflating: dist/torch-2.6.0a0+gitd1aa4ef-cp310-cp310-linux_x86_64.whl 2024-11-01T16:54:36.2248940Z creating: build/custom_test_artifacts/ 2024-11-01T16:54:36.2249729Z creating: build/custom_test_artifacts/custom-op-build/ 2024-11-01T16:54:36.2250569Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-11-01T16:54:36.2251537Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-11-01T16:54:36.2254024Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-11-01T16:54:36.2255124Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-11-01T16:54:36.2256236Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-11-01T16:54:36.2257388Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-11-01T16:54:36.2258539Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-11-01T16:54:36.2259833Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-11-01T16:54:36.2261381Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-11-01T16:54:36.2262564Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-11-01T16:54:36.2263726Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-11-01T16:54:36.2265104Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-11-01T16:54:36.2266459Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-11-01T16:54:36.2267813Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-11-01T16:54:36.2269169Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-11-01T16:54:36.2270529Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-11-01T16:54:36.2271904Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-11-01T16:54:36.2273217Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-11-01T16:54:36.2274324Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-11-01T16:54:36.2311140Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-11-01T16:54:36.2349166Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-11-01T16:54:36.2353230Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-11-01T16:54:36.2393061Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-11-01T16:54:36.2397790Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-11-01T16:54:36.2400849Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-11-01T16:54:36.2402669Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-11-01T16:54:36.2404410Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-11-01T16:54:36.2406345Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-11-01T16:54:36.2408069Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-11-01T16:54:36.2409929Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-11-01T16:54:36.2411604Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-11-01T16:54:36.2413176Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-11-01T16:54:36.2414682Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-11-01T16:54:36.2416160Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-11-01T16:54:36.2417671Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-11-01T16:54:36.2419119Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-11-01T16:54:36.2420590Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-11-01T16:54:36.2460946Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-11-01T16:54:36.2520559Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-11-01T16:54:36.2522210Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-11-01T16:54:36.2523424Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-11-01T16:54:36.2524456Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-11-01T16:54:36.2525542Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-11-01T16:54:36.2526646Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-11-01T16:54:36.2527874Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-11-01T16:54:36.2529261Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-11-01T16:54:36.2530621Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-11-01T16:54:36.2531866Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-11-01T16:54:36.2533266Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-11-01T16:54:36.2534528Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-11-01T16:54:36.2535793Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-11-01T16:54:36.2537123Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-11-01T16:54:36.2538281Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-11-01T16:54:36.2546192Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-11-01T16:54:36.2687568Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-11-01T16:54:36.2690914Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-11-01T16:54:36.2692246Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-11-01T16:54:36.2693797Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-11-01T16:54:36.2695364Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-11-01T16:54:36.2696700Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-11-01T16:54:36.2697938Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-11-01T16:54:36.2699208Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-11-01T16:54:36.2700475Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-11-01T16:54:36.2701731Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-11-01T16:54:36.2702968Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-11-01T16:54:36.2711922Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-11-01T16:54:36.2793364Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-11-01T16:54:36.2797285Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-11-01T16:54:36.2800311Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-11-01T16:54:36.2801643Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-11-01T16:54:36.2802720Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-11-01T16:54:36.2803765Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-11-01T16:54:36.2804823Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-11-01T16:54:36.2805812Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-11-01T16:54:36.2806688Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-11-01T16:54:36.2807619Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-11-01T16:54:36.2922173Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-11-01T16:54:36.2985011Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-11-01T16:54:36.2987337Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-11-01T16:54:36.2989471Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-11-01T16:54:36.2991153Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-11-01T16:54:36.2992556Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-11-01T16:54:36.2993648Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-11-01T16:54:36.2994760Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-11-01T16:54:36.2995943Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-11-01T16:54:36.2997089Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-11-01T16:54:36.2998492Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-11-01T16:54:36.2999777Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-11-01T16:54:36.3000934Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-11-01T16:54:36.3002060Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-11-01T16:54:36.3003409Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-11-01T16:54:36.3004769Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-11-01T16:54:36.3006097Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-11-01T16:54:36.3007612Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-11-01T16:54:36.3008970Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-11-01T16:54:36.3010585Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-11-01T16:54:36.3011811Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-11-01T16:54:36.3013005Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-11-01T16:54:36.3046846Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-11-01T16:54:36.3084857Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-11-01T16:54:36.3089601Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-11-01T16:54:36.3130070Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-11-01T16:54:36.3131853Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-11-01T16:54:36.3133830Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-11-01T16:54:36.3135640Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-11-01T16:54:36.3137344Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-11-01T16:54:36.3139027Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-11-01T16:54:36.3140741Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-11-01T16:54:36.3142460Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-11-01T16:54:36.3144260Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-11-01T16:54:36.3145796Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-11-01T16:54:36.3147361Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-11-01T16:54:36.3148732Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-11-01T16:54:36.3150088Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-11-01T16:54:36.3151431Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-11-01T16:54:36.3152799Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-11-01T16:54:36.3201675Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-11-01T16:54:36.3262755Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-11-01T16:54:36.3266647Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-11-01T16:54:36.3269671Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-11-01T16:54:36.3270700Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-11-01T16:54:36.3271734Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-11-01T16:54:36.3273053Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-11-01T16:54:36.3274320Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-11-01T16:54:36.3275742Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-11-01T16:54:36.3277116Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-11-01T16:54:36.3278456Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-11-01T16:54:36.3279711Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-11-01T16:54:36.3280939Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-11-01T16:54:36.3282192Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-11-01T16:54:36.3283446Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-11-01T16:54:36.3284691Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-11-01T16:54:36.3286538Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-11-01T16:54:36.3354257Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-11-01T16:54:36.3358177Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-11-01T16:54:36.3360597Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-11-01T16:54:36.3361731Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-11-01T16:54:36.3362855Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-11-01T16:54:36.3363857Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-11-01T16:54:36.3364858Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-11-01T16:54:36.3365806Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-11-01T16:54:36.3366660Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-11-01T16:54:36.3367554Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-11-01T16:54:36.3434981Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-11-01T16:54:36.3436365Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-11-01T16:54:36.3437293Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-11-01T16:54:36.3438361Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-11-01T16:54:36.3439636Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-11-01T16:54:36.3440855Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-11-01T16:54:36.3442084Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-11-01T16:54:36.3443338Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-11-01T16:54:36.3444577Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-11-01T16:54:36.3445974Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-11-01T16:54:36.3447393Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-11-01T16:54:36.3448667Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-11-01T16:54:36.3449919Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-11-01T16:54:36.3451587Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-11-01T16:54:36.3453051Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-11-01T16:54:36.3454529Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-11-01T16:54:36.3455864Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-11-01T16:54:36.3457229Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-11-01T16:54:36.3458593Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-11-01T16:54:36.3459804Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-11-01T16:54:36.3460981Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-11-01T16:54:36.3475114Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-11-01T16:54:36.3513756Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-11-01T16:54:36.3519124Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-11-01T16:54:36.3557796Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-11-01T16:54:36.3559680Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-11-01T16:54:36.3561698Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-11-01T16:54:36.3563459Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-11-01T16:54:36.3565123Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-11-01T16:54:36.3566766Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-11-01T16:54:36.3568414Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-11-01T16:54:36.3570042Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-11-01T16:54:36.3571642Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-11-01T16:54:36.3573150Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-11-01T16:54:36.3574595Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-11-01T16:54:36.3576018Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-11-01T16:54:36.3577461Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-11-01T16:54:36.3578863Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-11-01T16:54:36.3580290Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-11-01T16:54:36.3628552Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-11-01T16:54:36.3688158Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-11-01T16:54:36.3690725Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-11-01T16:54:36.3692010Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-11-01T16:54:36.3693120Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-11-01T16:54:36.3694278Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-11-01T16:54:36.3695487Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-11-01T16:54:36.3696863Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-11-01T16:54:36.3698384Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-11-01T16:54:36.3699945Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-11-01T16:54:36.3701292Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-11-01T16:54:36.3702680Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-11-01T16:54:36.3704204Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-11-01T16:54:36.3705620Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-11-01T16:54:36.3707026Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-11-01T16:54:36.3708407Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-11-01T16:54:36.3710120Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-11-01T16:54:36.3817458Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-11-01T16:54:36.3820504Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-11-01T16:54:36.3821968Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-11-01T16:54:36.3823596Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-11-01T16:54:36.3825149Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-11-01T16:54:36.3826613Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-11-01T16:54:36.3828120Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-11-01T16:54:36.3829712Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-11-01T16:54:36.3831109Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-11-01T16:54:36.3832500Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-11-01T16:54:36.3833873Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-11-01T16:54:36.3841557Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-11-01T16:54:36.3898157Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-11-01T16:54:36.3902421Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-11-01T16:54:36.3904373Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-11-01T16:54:36.3905503Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-11-01T16:54:36.3906563Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-11-01T16:54:36.3907622Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-11-01T16:54:36.3908672Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-11-01T16:54:36.3909751Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-11-01T16:54:36.3910910Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-11-01T16:54:36.3911939Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-11-01T16:54:36.4002583Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-11-01T16:54:36.4046227Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-11-01T16:54:36.4048214Z creating: build/lib/ 2024-11-01T16:54:36.4054881Z inflating: build/lib/libpthreadpool.a 2024-11-01T16:54:36.4061909Z inflating: build/lib/libcpuinfo_internals.a 2024-11-01T16:54:36.4069936Z inflating: build/lib/libcpuinfo.a 2024-11-01T16:54:36.4070674Z inflating: build/lib/libclog.a 2024-11-01T16:54:36.4072793Z inflating: build/lib/libnnpack_reference_layers.a 2024-11-01T16:54:36.4163567Z inflating: build/lib/libprotobuf-lite.a 2024-11-01T16:54:36.4608050Z inflating: build/lib/libprotobuf.a 2024-11-01T16:54:36.4672297Z inflating: build/lib/libgtest.a 2024-11-01T16:54:36.4747436Z inflating: build/lib/libbenchmark.a 2024-11-01T16:54:36.4810488Z inflating: build/lib/libasmjit.a 2024-11-01T16:54:36.4816899Z inflating: build/lib/libittnotify.a 2024-11-01T16:54:36.4836757Z inflating: build/lib/libnnpack.a 2024-11-01T16:54:36.4863989Z inflating: build/lib/libtensorpipe_uv.a 2024-11-01T16:54:36.4990593Z inflating: build/lib/libgloo.a 2024-11-01T16:54:36.5011765Z inflating: build/lib/libfmt.a 2024-11-01T16:54:36.5013148Z inflating: build/lib/libcaffe2_nvrtc.so 2024-11-01T16:54:36.5111166Z inflating: build/lib/libc10.so 2024-11-01T16:54:36.5112602Z inflating: build/lib/libtorch_global_deps.so 2024-11-01T16:54:36.5132202Z inflating: build/lib/libpytorch_qnnpack.a 2024-11-01T16:54:36.5622377Z inflating: build/lib/libprotoc.a 2024-11-01T16:54:36.5640771Z inflating: build/lib/libgmock.a 2024-11-01T16:54:36.5642189Z inflating: build/lib/libbenchmark_main.a 2024-11-01T16:54:36.5643556Z inflating: build/lib/libgtest_main.a 2024-11-01T16:54:37.5489812Z inflating: build/lib/libdnnl.a 2024-11-01T16:54:37.6045681Z inflating: build/lib/libtensorpipe.a 2024-11-01T16:54:37.6104426Z inflating: build/lib/libc10_cuda.so 2024-11-01T16:54:37.6105802Z inflating: build/lib/libgmock_main.a 2024-11-01T16:54:37.7304887Z inflating: build/lib/libfbgemm.a 2024-11-01T16:54:37.7803588Z inflating: build/lib/libkineto.a 2024-11-01T16:54:37.8055504Z inflating: build/lib/libtensorpipe_cuda.a 2024-11-01T16:54:37.8247267Z inflating: build/lib/libXNNPACK.a 2024-11-01T16:54:37.8288815Z inflating: build/lib/libonnx_proto.a 2024-11-01T16:54:37.8657028Z inflating: build/lib/libgloo_cuda.a 2024-11-01T16:54:37.9365381Z inflating: build/lib/libonnx.a 2024-11-01T16:54:40.3571508Z inflating: build/lib/libtorch_cpu.so 2024-11-01T16:54:40.3574346Z inflating: build/lib/libshm.so 2024-11-01T16:54:40.3579267Z inflating: build/lib/libunbox_lib.a 2024-11-01T16:54:41.9990360Z inflating: build/lib/libtorch_cuda.so 2024-11-01T16:54:41.9991981Z inflating: build/lib/libtorch.so 2024-11-01T16:54:41.9992684Z inflating: build/lib/libc10d_cuda_test.so 2024-11-01T16:54:42.8087336Z inflating: build/lib/libtorch_cuda_linalg.so 2024-11-01T16:54:42.8107887Z inflating: build/lib/libjitbackend_test.so 2024-11-01T16:54:42.8176738Z inflating: build/lib/libtorchbind_test.so 2024-11-01T16:54:42.8200759Z inflating: build/lib/libaoti_custom_ops.so 2024-11-01T16:54:42.8226692Z inflating: build/lib/libbackend_with_compiler.so 2024-11-01T16:54:43.0185458Z inflating: build/lib/libtorch_python.so 2024-11-01T16:54:43.0220617Z inflating: build/lib/libnnapi_backend.so 2024-11-01T16:54:43.0221276Z creating: build/bin/ 2024-11-01T16:54:43.0271610Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-11-01T16:54:43.0322985Z inflating: build/bin/c10_DeviceGuard_test 2024-11-01T16:54:43.0375832Z inflating: build/bin/c10_Device_test 2024-11-01T16:54:43.0434622Z inflating: build/bin/c10_DispatchKeySet_test 2024-11-01T16:54:43.0487834Z inflating: build/bin/c10_Scalar_test 2024-11-01T16:54:43.0536676Z inflating: build/bin/c10_StreamGuard_test 2024-11-01T16:54:43.0591087Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-11-01T16:54:43.0641742Z inflating: build/bin/c10_SymInt_test 2024-11-01T16:54:43.0698025Z inflating: build/bin/c10_InlineStreamGuard_test 2024-11-01T16:54:43.0754880Z inflating: build/bin/c10_SizesAndStrides_test 2024-11-01T16:54:43.0826028Z inflating: build/bin/c10_cow_test 2024-11-01T16:54:43.0879678Z inflating: build/bin/c10_Bitset_test 2024-11-01T16:54:43.0930011Z inflating: build/bin/c10_ConstexprCrc_test 2024-11-01T16:54:43.0980102Z inflating: build/bin/c10_DeadlockDetection_test 2024-11-01T16:54:43.1030847Z inflating: build/bin/c10_Half_test 2024-11-01T16:54:43.1087723Z inflating: build/bin/c10_LeftRight_test 2024-11-01T16:54:43.1143679Z inflating: build/bin/c10_Metaprogramming_test 2024-11-01T16:54:43.1196474Z inflating: build/bin/c10_NetworkFlow_test 2024-11-01T16:54:43.1247263Z inflating: build/bin/c10_Synchronized_test 2024-11-01T16:54:43.1302781Z inflating: build/bin/c10_ThreadLocal_test 2024-11-01T16:54:43.1355334Z inflating: build/bin/c10_TypeIndex_test 2024-11-01T16:54:43.1406301Z inflating: build/bin/c10_TypeList_test 2024-11-01T16:54:43.1455508Z inflating: build/bin/c10_TypeTraits_test 2024-11-01T16:54:43.1507017Z inflating: build/bin/c10_accumulate_test 2024-11-01T16:54:43.1557572Z inflating: build/bin/c10_bit_cast_test 2024-11-01T16:54:43.1612138Z inflating: build/bin/c10_bfloat16_test 2024-11-01T16:54:43.1667894Z inflating: build/bin/c10_complex_test 2024-11-01T16:54:43.1724958Z inflating: build/bin/c10_complex_math_test 2024-11-01T16:54:43.1777286Z inflating: build/bin/c10_exception_test 2024-11-01T16:54:43.1828424Z inflating: build/bin/c10_flags_test 2024-11-01T16:54:43.1879615Z inflating: build/bin/c10_generic_math_test 2024-11-01T16:54:43.2040677Z inflating: build/bin/c10_intrusive_ptr_test 2024-11-01T16:54:43.2091204Z inflating: build/bin/c10_irange_test 2024-11-01T16:54:43.2145686Z inflating: build/bin/c10_lazy_test 2024-11-01T16:54:43.2202446Z inflating: build/bin/c10_logging_test 2024-11-01T16:54:43.2277109Z inflating: build/bin/c10_optional_test 2024-11-01T16:54:43.2338699Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-11-01T16:54:43.2393369Z inflating: build/bin/c10_registry_test 2024-11-01T16:54:43.2541059Z inflating: build/bin/c10_small_vector_test 2024-11-01T16:54:43.2592310Z inflating: build/bin/c10_ssize_test 2024-11-01T16:54:43.2645252Z inflating: build/bin/c10_string_util_test 2024-11-01T16:54:43.2704271Z inflating: build/bin/c10_string_view_test 2024-11-01T16:54:43.2754892Z inflating: build/bin/c10_tempfile_test 2024-11-01T16:54:43.2810795Z inflating: build/bin/c10_typeid_test 2024-11-01T16:54:43.2861345Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-11-01T16:54:43.3314796Z inflating: build/bin/protoc-3.13.0.0 2024-11-01T16:54:43.3766619Z inflating: build/bin/protoc 2024-11-01T16:54:43.3820249Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-11-01T16:54:43.3874758Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-11-01T16:54:43.3929763Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-11-01T16:54:43.3982647Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-11-01T16:54:43.4035423Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-11-01T16:54:43.4083799Z inflating: build/bin/c10_cuda_CUDATest 2024-11-01T16:54:43.4138991Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-11-01T16:54:43.4190349Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-11-01T16:54:43.4563241Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-11-01T16:54:43.4945218Z inflating: build/bin/vec_test_all_types_AVX512 2024-11-01T16:54:43.5336898Z inflating: build/bin/vec_test_all_types_AVX2 2024-11-01T16:54:43.5390756Z inflating: build/bin/HashStoreTest 2024-11-01T16:54:43.5446203Z inflating: build/bin/TCPStoreTest 2024-11-01T16:54:43.5498909Z inflating: build/bin/FileStoreTest 2024-11-01T16:54:43.5550842Z inflating: build/bin/BackoffTest 2024-11-01T16:54:43.5565767Z inflating: build/bin/ProcessGroupMPITest 2024-11-01T16:54:43.5568304Z inflating: build/bin/torch_shm_manager 2024-11-01T16:54:43.5624802Z inflating: build/bin/test_edge_op_registration 2024-11-01T16:54:43.5627424Z inflating: build/bin/example_allreduce 2024-11-01T16:54:43.5679771Z inflating: build/bin/Dimname_test 2024-11-01T16:54:43.5753497Z inflating: build/bin/Dict_test 2024-11-01T16:54:43.5817690Z inflating: build/bin/MaybeOwned_test 2024-11-01T16:54:43.5878080Z inflating: build/bin/apply_utils_test 2024-11-01T16:54:43.5936908Z inflating: build/bin/NamedTensor_test 2024-11-01T16:54:43.5995228Z inflating: build/bin/atest 2024-11-01T16:54:43.6060062Z inflating: build/bin/basic 2024-11-01T16:54:43.6115268Z inflating: build/bin/broadcast_test 2024-11-01T16:54:43.6168558Z inflating: build/bin/cpu_profiling_allocator_test 2024-11-01T16:54:43.6227019Z inflating: build/bin/cpu_generator_test 2024-11-01T16:54:43.6278869Z inflating: build/bin/cpu_allocator_test 2024-11-01T16:54:43.6329739Z inflating: build/bin/dispatch_key_set_test 2024-11-01T16:54:43.6422784Z inflating: build/bin/cpu_rng_test 2024-11-01T16:54:43.6473122Z inflating: build/bin/dlconvertor_test 2024-11-01T16:54:43.6531751Z inflating: build/bin/extension_backend_test 2024-11-01T16:54:43.6586834Z inflating: build/bin/half_test 2024-11-01T16:54:43.6682323Z inflating: build/bin/ivalue_test 2024-11-01T16:54:43.6732789Z inflating: build/bin/lazy_tensor_test 2024-11-01T16:54:43.6787017Z inflating: build/bin/math_kernel_test 2024-11-01T16:54:43.6841636Z inflating: build/bin/memory_format_test 2024-11-01T16:54:43.6895702Z inflating: build/bin/memory_overlapping_test 2024-11-01T16:54:43.6947014Z inflating: build/bin/operator_name_test 2024-11-01T16:54:43.7002555Z inflating: build/bin/native_test 2024-11-01T16:54:43.7056273Z inflating: build/bin/mobile_memory_cleanup 2024-11-01T16:54:43.7107456Z inflating: build/bin/operators_test 2024-11-01T16:54:43.7159836Z inflating: build/bin/packedtensoraccessor_test 2024-11-01T16:54:43.7226316Z inflating: build/bin/pow_test 2024-11-01T16:54:43.7283112Z inflating: build/bin/quantized_test 2024-11-01T16:54:43.7334065Z inflating: build/bin/reduce_ops_test 2024-11-01T16:54:43.7385865Z inflating: build/bin/reportMemoryUsage_test 2024-11-01T16:54:43.7443789Z inflating: build/bin/scalar_tensor_test 2024-11-01T16:54:43.7501278Z inflating: build/bin/scalar_test 2024-11-01T16:54:43.7553760Z inflating: build/bin/StorageUtils_test 2024-11-01T16:54:43.7606274Z inflating: build/bin/stride_properties_test 2024-11-01T16:54:43.7684446Z inflating: build/bin/tensor_iterator_test 2024-11-01T16:54:43.7739069Z inflating: build/bin/test_parallel 2024-11-01T16:54:43.7741155Z inflating: build/bin/thread_init_test 2024-11-01T16:54:43.7797151Z inflating: build/bin/type_ptr_test 2024-11-01T16:54:43.7857147Z inflating: build/bin/type_test 2024-11-01T16:54:43.7857839Z inflating: build/bin/verify_api_visibility 2024-11-01T16:54:43.7912392Z inflating: build/bin/undefined_tensor_test 2024-11-01T16:54:43.7981334Z inflating: build/bin/legacy_vmap_test 2024-11-01T16:54:43.8033851Z inflating: build/bin/weakref_test 2024-11-01T16:54:43.8085186Z inflating: build/bin/wrapdim_test 2024-11-01T16:54:43.8190659Z inflating: build/bin/List_test 2024-11-01T16:54:43.8243840Z inflating: build/bin/xla_tensor_test 2024-11-01T16:54:43.8304061Z inflating: build/bin/IListRef_test 2024-11-01T16:54:43.8425420Z inflating: build/bin/kernel_function_legacy_test 2024-11-01T16:54:43.8520401Z inflating: build/bin/kernel_function_test 2024-11-01T16:54:43.8588048Z inflating: build/bin/KernelFunction_test 2024-11-01T16:54:43.8714528Z inflating: build/bin/kernel_lambda_legacy_test 2024-11-01T16:54:43.8815822Z inflating: build/bin/kernel_lambda_test 2024-11-01T16:54:43.8876890Z inflating: build/bin/kernel_stackbased_test 2024-11-01T16:54:43.8929881Z inflating: build/bin/CppSignature_test 2024-11-01T16:54:43.9025863Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-11-01T16:54:43.9081236Z inflating: build/bin/backend_fallback_test 2024-11-01T16:54:43.9131205Z inflating: build/bin/op_allowlist_test 2024-11-01T16:54:43.9424208Z inflating: build/bin/op_registration_test 2024-11-01T16:54:43.9487594Z inflating: build/bin/inline_container_test 2024-11-01T16:54:43.9540635Z inflating: build/bin/cuda_apply_test 2024-11-01T16:54:43.9592659Z inflating: build/bin/cuda_allocator_test 2024-11-01T16:54:43.9647440Z inflating: build/bin/cuda_caching_host_allocator_test 2024-11-01T16:54:43.9706026Z inflating: build/bin/cuda_atomic_ops_test 2024-11-01T16:54:43.9775016Z inflating: build/bin/cuda_complex_math_test 2024-11-01T16:54:43.9833663Z inflating: build/bin/cuda_complex_test 2024-11-01T16:54:43.9883882Z inflating: build/bin/cuda_device_test 2024-11-01T16:54:43.9943514Z inflating: build/bin/cuda_cub_test 2024-11-01T16:54:43.9994211Z inflating: build/bin/cuda_dlconvertor_test 2024-11-01T16:54:44.0046232Z inflating: build/bin/cuda_integer_divider_test 2024-11-01T16:54:44.0110956Z inflating: build/bin/cuda_distributions_test 2024-11-01T16:54:44.0167348Z inflating: build/bin/cuda_generator_test 2024-11-01T16:54:44.0217577Z inflating: build/bin/cuda_half_test 2024-11-01T16:54:44.0267795Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-11-01T16:54:44.0320267Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-11-01T16:54:44.0369707Z inflating: build/bin/cuda_optional_test 2024-11-01T16:54:44.0432491Z inflating: build/bin/cuda_stream_test 2024-11-01T16:54:44.0484496Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-11-01T16:54:44.0534764Z inflating: build/bin/cuda_cudnn_test 2024-11-01T16:54:44.0586829Z inflating: build/bin/cuda_vectorized_test 2024-11-01T16:54:44.0653243Z inflating: build/bin/ProcessGroupGlooTest 2024-11-01T16:54:44.0668087Z inflating: build/bin/tutorial_tensorexpr 2024-11-01T16:54:44.0727138Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-11-01T16:54:44.0791112Z inflating: build/bin/ProcessGroupNCCLTest 2024-11-01T16:54:44.0854409Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-11-01T16:54:44.0910160Z inflating: build/bin/test_dist_autograd 2024-11-01T16:54:44.0911449Z inflating: build/bin/parallel_benchmark 2024-11-01T16:54:44.0981317Z inflating: build/bin/test_cpp_rpc 2024-11-01T16:54:44.1049352Z inflating: build/bin/test_mobile_nnc 2024-11-01T16:54:44.1058562Z inflating: build/bin/aot_model_compiler_test 2024-11-01T16:54:44.1393972Z inflating: build/bin/test_lazy 2024-11-01T16:54:44.2205009Z inflating: build/bin/test_tensorexpr 2024-11-01T16:54:44.3353739Z inflating: build/bin/test_api 2024-11-01T16:54:44.3925098Z inflating: build/bin/test_jit 2024-11-01T16:54:44.3926892Z creating: .additional_ci_files/ 2024-11-01T16:54:44.4005165Z inflating: .additional_ci_files/test-times.json 2024-11-01T16:54:44.4314170Z inflating: .additional_ci_files/test-class-times.json 2024-11-01T16:54:44.4349190Z ##[group]Run rm artifacts.zip 2024-11-01T16:54:44.4349600Z rm artifacts.zip 2024-11-01T16:54:44.4356057Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:44.4356638Z env: 2024-11-01T16:54:44.4356951Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:44.4357463Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:44.4358024Z ##[endgroup] 2024-11-01T16:54:44.4953815Z ##[group]Run df -H 2024-11-01T16:54:44.4954149Z df -H 2024-11-01T16:54:44.4959059Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:44.4959609Z env: 2024-11-01T16:54:44.4959907Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:44.4960373Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:44.4960893Z ##[endgroup] 2024-11-01T16:54:44.4997590Z Filesystem Size Used Avail Use% Mounted on 2024-11-01T16:54:44.4999965Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-11-01T16:54:44.5000833Z tmpfs 203G 0 203G 0% /dev/shm 2024-11-01T16:54:44.5001384Z tmpfs 82G 11M 82G 1% /run 2024-11-01T16:54:44.5002227Z /dev/nvme0n1p1 161G 53G 109G 33% / 2024-11-01T16:54:44.5002712Z tmpfs 203G 13k 203G 1% /tmp 2024-11-01T16:54:44.5003228Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-11-01T16:54:44.5003747Z tmpfs 41G 0 41G 0% /run/user/0 2024-11-01T16:54:44.5047875Z Prepare all required actions 2024-11-01T16:54:44.5048330Z Getting action download info 2024-11-01T16:54:44.6975799Z ##[group]Run ./.github/actions/download-td-artifacts 2024-11-01T16:54:44.6976290Z with: 2024-11-01T16:54:44.6976545Z env: 2024-11-01T16:54:44.6976822Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:44.6977287Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:44.6977800Z ##[endgroup] 2024-11-01T16:54:44.7013625Z ##[group]Run seemethere/download-artifact-s3@v4 2024-11-01T16:54:44.7014108Z with: 2024-11-01T16:54:44.7014381Z name: td_results 2024-11-01T16:54:44.7014713Z s3-bucket: gha-artifacts 2024-11-01T16:54:44.7015070Z region: us-east-1 2024-11-01T16:54:44.7015383Z env: 2024-11-01T16:54:44.7015707Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:44.7016186Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:44.7016707Z ##[endgroup] 2024-11-01T16:54:45.1629476Z (node:83504) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-11-01T16:54:45.1630312Z 2024-11-01T16:54:45.1630632Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-11-01T16:54:45.1631490Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-11-01T16:54:45.1632530Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-11-01T16:54:45.2488367Z Found 1 objects with prefix pytorch/pytorch/11632514903/td_results/ 2024-11-01T16:54:45.2489533Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-11-01T16:54:45.3168207Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-11-01T16:54:45.3171769Z Artifact download has finished successfully 2024-11-01T16:54:45.3358045Z ##[group]Run mkdir -p .additional_ci_files 2024-11-01T16:54:45.3374047Z mkdir -p .additional_ci_files 2024-11-01T16:54:45.3374681Z mv td_results.json .additional_ci_files/td_results.json 2024-11-01T16:54:45.3381065Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:45.3381570Z env: 2024-11-01T16:54:45.3381842Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:45.3382302Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:45.3382967Z ##[endgroup] 2024-11-01T16:54:45.3463702Z ##[group]Run .github/scripts/parse_ref.py 2024-11-01T16:54:45.3464185Z .github/scripts/parse_ref.py 2024-11-01T16:54:45.3468746Z shell: /usr/bin/bash -e {0} 2024-11-01T16:54:45.3469119Z env: 2024-11-01T16:54:45.3469402Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:45.3469872Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:45.3470394Z ##[endgroup] 2024-11-01T16:54:45.3742297Z Prepare all required actions 2024-11-01T16:54:45.3818316Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-11-01T16:54:45.3818831Z with: 2024-11-01T16:54:45.3819481Z github-token: *** 2024-11-01T16:54:45.3819806Z env: 2024-11-01T16:54:45.3820094Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:45.3820591Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:45.3821111Z ##[endgroup] 2024-11-01T16:54:45.3839131Z ##[group]Run set -eux 2024-11-01T16:54:45.3839468Z set -eux 2024-11-01T16:54:45.3840092Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-11-01T16:54:45.3844953Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:45.3845455Z env: 2024-11-01T16:54:45.3845728Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:45.3846190Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:45.3846870Z GITHUB_TOKEN: *** 2024-11-01T16:54:45.3847176Z ##[endgroup] 2024-11-01T16:54:45.3869017Z + python3 .github/scripts/get_workflow_job_id.py 11632514903 i-0fb054b0c783c31fa 2024-11-01T16:54:47.4393869Z setting job-id=32397131891 2024-11-01T16:54:47.4395002Z setting job-name=cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) 2024-11-01T16:54:47.4659741Z Prepare all required actions 2024-11-01T16:54:47.4660255Z Getting action download info 2024-11-01T16:54:47.5815422Z ##[group]Run ./.github/actions/filter-test-configs 2024-11-01T16:54:47.5815951Z with: 2024-11-01T16:54:47.5816481Z github-token: *** 2024-11-01T16:54:47.5817347Z test-matrix: {"include": [{"config": "pr_time_benchmarks", "shard": 1, "num_shards": 1, "runner": "linux.g4dn.metal.nvidia.gpu"}]} 2024-11-01T16:54:47.5818682Z job-name: cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) 2024-11-01T16:54:47.5819519Z env: 2024-11-01T16:54:47.5819846Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:47.5820515Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:47.5821055Z ##[endgroup] 2024-11-01T16:54:47.5866892Z ##[group]Run nick-fields/retry@v3.0.0 2024-11-01T16:54:47.5867288Z with: 2024-11-01T16:54:47.5867728Z shell: bash 2024-11-01T16:54:47.5868059Z timeout_minutes: 10 2024-11-01T16:54:47.5868460Z max_attempts: 5 2024-11-01T16:54:47.5868776Z retry_wait_seconds: 30 2024-11-01T16:54:47.5870053Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-11-01T16:54:47.5871425Z polling_interval_seconds: 1 2024-11-01T16:54:47.5871827Z warning_on_retry: true 2024-11-01T16:54:47.5872198Z continue_on_error: false 2024-11-01T16:54:47.5872552Z env: 2024-11-01T16:54:47.5872824Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:47.5873336Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:47.5874093Z GITHUB_TOKEN: *** 2024-11-01T16:54:47.5874460Z ##[endgroup] 2024-11-01T16:54:47.6778353Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-11-01T16:54:47.9148664Z Defaulting to user installation because normal site-packages is not writeable 2024-11-01T16:54:47.9300519Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2024-11-01T16:54:47.9304343Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2024-11-01T16:54:47.9418853Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2024.8.30) 2024-11-01T16:54:47.9420923Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-11-01T16:54:47.9430937Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-11-01T16:54:47.9444364Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2024-11-01T16:54:48.6617124Z Command completed after 1 attempt(s). 2024-11-01T16:54:48.6676713Z ##[group]Run set -x 2024-11-01T16:54:48.6677074Z set -x 2024-11-01T16:54:48.6677389Z  2024-11-01T16:54:48.6678003Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-11-01T16:54:48.6678742Z # in runner workspace 2024-11-01T16:54:48.6679300Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-11-01T16:54:48.6686454Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:48.6686976Z env: 2024-11-01T16:54:48.6687275Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:48.6687766Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:48.6688301Z ##[endgroup] 2024-11-01T16:54:48.6716568Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-11-01T16:54:48.6983540Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-11-01T16:54:48.6984126Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-11-01T16:54:48.6984600Z echo "Job name: ${JOB_NAME}" 2024-11-01T16:54:48.6985017Z  2024-11-01T16:54:48.6985779Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-11-01T16:54:48.6986511Z # in runner workspace 2024-11-01T16:54:48.6987120Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-11-01T16:54:48.6987812Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-11-01T16:54:48.6988270Z  --job-name "${JOB_NAME}" \ 2024-11-01T16:54:48.6989204Z  --test-matrix "{"include": [{"config": "pr_time_benchmarks", "shard": 1, "num_shards": 1, "runner": "linux.g4dn.metal.nvidia.gpu"}]}" \ 2024-11-01T16:54:48.6990168Z  --selected-test-configs "" \ 2024-11-01T16:54:48.6990636Z  --pr-number "${PR_NUMBER}" \ 2024-11-01T16:54:48.6991065Z  --tag "${TAG}" \ 2024-11-01T16:54:48.6991466Z  --event-name "${EVENT_NAME}" \ 2024-11-01T16:54:48.6991909Z  --schedule "${SCHEDULE}" \ 2024-11-01T16:54:48.6992346Z  --branch "${HEAD_BRANCH}" 2024-11-01T16:54:48.6997914Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:48.6998431Z env: 2024-11-01T16:54:48.6998709Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:48.6999183Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:48.6999929Z GITHUB_TOKEN: *** 2024-11-01T16:54:48.7000597Z JOB_NAME: cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) 2024-11-01T16:54:48.7001371Z PR_NUMBER: 138766 2024-11-01T16:54:48.7001684Z TAG: 2024-11-01T16:54:48.7001970Z EVENT_NAME: pull_request 2024-11-01T16:54:48.7002327Z SCHEDULE: 2024-11-01T16:54:48.7002602Z HEAD_BRANCH: 2024-11-01T16:54:48.7002899Z ##[endgroup] 2024-11-01T16:54:48.7026303Z Workflow: pull 2024-11-01T16:54:48.7028779Z Job name: cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) 2024-11-01T16:54:48.9204799Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-11-01T16:54:49.0764695Z ##[group]Run echo "Filtered matrix:" 2024-11-01T16:54:49.0765154Z echo "Filtered matrix:" 2024-11-01T16:54:49.0765993Z echo "{"include": [{"config": "pr_time_benchmarks", "shard": 1, "num_shards": 1, "runner": "linux.g4dn.metal.nvidia.gpu"}]}" 2024-11-01T16:54:49.0766982Z  2024-11-01T16:54:49.0767247Z echo 2024-11-01T16:54:49.0767623Z echo "Is the current job unstable? False" 2024-11-01T16:54:49.0768089Z  2024-11-01T16:54:49.0768353Z echo 2024-11-01T16:54:49.0768696Z echo "Is keep-going label set? False" 2024-11-01T16:54:49.0769144Z  2024-11-01T16:54:49.0769403Z echo 2024-11-01T16:54:49.0769712Z echo "Renabled issues? " 2024-11-01T16:54:49.0774436Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:49.0774949Z env: 2024-11-01T16:54:49.0775205Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:49.0775668Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:49.0776161Z ##[endgroup] 2024-11-01T16:54:49.0798391Z Filtered matrix: 2024-11-01T16:54:49.0799648Z {include: [{config: pr_time_benchmarks, shard: 1, num_shards: 1, runner: linux.g4dn.metal.nvidia.gpu}]} 2024-11-01T16:54:49.0800367Z 2024-11-01T16:54:49.0800565Z Is the current job unstable? False 2024-11-01T16:54:49.0800905Z 2024-11-01T16:54:49.0801190Z Is keep-going label set? False 2024-11-01T16:54:49.0801480Z 2024-11-01T16:54:49.0801612Z Renabled issues? 2024-11-01T16:54:49.0849650Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-11-01T16:54:49.0850419Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-11-01T16:54:49.0855744Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T16:54:49.0856288Z env: 2024-11-01T16:54:49.0856585Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:49.0857047Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:49.0857580Z JOB_TIMEOUT: 240 2024-11-01T16:54:49.0857905Z ##[endgroup] 2024-11-01T16:54:49.0944960Z ##[group]Run # Fetch aws credential from IMDs 2024-11-01T16:54:49.0945559Z # Fetch aws credential from IMDs 2024-11-01T16:54:49.0946180Z eval "$(python3 .github/scripts/get_aws_session_tokens.py)" 2024-11-01T16:54:49.0946755Z set -x 2024-11-01T16:54:49.0947050Z  2024-11-01T16:54:49.0947399Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-11-01T16:54:49.0947955Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-11-01T16:54:49.0948532Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-11-01T16:54:49.0949059Z  TEST_COMMAND=.ci/onnx/test.sh 2024-11-01T16:54:49.0949472Z else 2024-11-01T16:54:49.0949794Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-11-01T16:54:49.0950220Z fi 2024-11-01T16:54:49.0950482Z  2024-11-01T16:54:49.0950952Z # detached container should get cleaned up by teardown_ec2_linux 2024-11-01T16:54:49.0951777Z # TODO: Stop building test binaries as part of the build phase 2024-11-01T16:54:49.0952478Z # Used for GPU_FLAG since that doesn't play nice 2024-11-01T16:54:49.0953037Z # shellcheck disable=SC2086,SC2090 2024-11-01T16:54:49.0953490Z container_name=$(docker run \ 2024-11-01T16:54:49.0953915Z  ${GPU_FLAG:-} \ 2024-11-01T16:54:49.0954330Z  ${SCCACHE_SERVER_PORT_DOCKER_FLAG:-} \ 2024-11-01T16:54:49.0954810Z  -e BUILD_ENVIRONMENT \ 2024-11-01T16:54:49.0955209Z  -e PR_NUMBER \ 2024-11-01T16:54:49.0955558Z  -e GITHUB_ACTIONS \ 2024-11-01T16:54:49.0955946Z  -e GITHUB_REPOSITORY \ 2024-11-01T16:54:49.0956350Z  -e GITHUB_WORKFLOW \ 2024-11-01T16:54:49.0956733Z  -e GITHUB_JOB \ 2024-11-01T16:54:49.0957100Z  -e GITHUB_RUN_ID \ 2024-11-01T16:54:49.0957480Z  -e GITHUB_RUN_NUMBER \ 2024-11-01T16:54:49.0957873Z  -e GITHUB_RUN_ATTEMPT \ 2024-11-01T16:54:49.0958271Z  -e JOB_ID \ 2024-11-01T16:54:49.0958608Z  -e JOB_NAME \ 2024-11-01T16:54:49.0958953Z  -e BASE_SHA \ 2024-11-01T16:54:49.0959292Z  -e BRANCH \ 2024-11-01T16:54:49.0959610Z  -e SHA1 \ 2024-11-01T16:54:49.0959941Z  -e AWS_DEFAULT_REGION \ 2024-11-01T16:54:49.0960499Z  -e AWS_ACCESS_KEY_ID \ 2024-11-01T16:54:49.0960907Z  -e AWS_SECRET_ACCESS_KEY \ 2024-11-01T16:54:49.0961327Z  -e AWS_SESSION_TOKEN \ 2024-11-01T16:54:49.0961706Z  -e IN_WHEEL_TEST \ 2024-11-01T16:54:49.0962079Z  -e SHARD_NUMBER \ 2024-11-01T16:54:49.0962449Z  -e TEST_CONFIG \ 2024-11-01T16:54:49.0962818Z  -e NUM_TEST_SHARDS \ 2024-11-01T16:54:49.0963209Z  -e REENABLED_ISSUES \ 2024-11-01T16:54:49.0963622Z  -e CONTINUE_THROUGH_ERROR \ 2024-11-01T16:54:49.0964036Z  -e VERBOSE_TEST_LOGS \ 2024-11-01T16:54:49.0964434Z  -e TEST_SHOWLOCALS \ 2024-11-01T16:54:49.0964833Z  -e NO_TEST_TIMEOUT \ 2024-11-01T16:54:49.0965203Z  -e NO_TD \ 2024-11-01T16:54:49.0965534Z  -e TD_DISTRIBUTED \ 2024-11-01T16:54:49.0965900Z  -e PR_LABELS \ 2024-11-01T16:54:49.0966303Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-11-01T16:54:49.0966772Z  -e SCCACHE_BUCKET \ 2024-11-01T16:54:49.0967153Z  -e SCCACHE_REGION \ 2024-11-01T16:54:49.0967548Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-11-01T16:54:49.0967958Z  -e XLA_CUDA \ 2024-11-01T16:54:49.0968345Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-11-01T16:54:49.0968865Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-11-01T16:54:49.0969399Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-11-01T16:54:49.0969925Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-11-01T16:54:49.0970406Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-11-01T16:54:49.0970863Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-11-01T16:54:49.0971385Z  -e DASHBOARD_TAG \ 2024-11-01T16:54:49.0971774Z  -e IS_A100_RUNNER \ 2024-11-01T16:54:49.0972168Z  -e ARTIFACTS_FILE_SUFFIX \ 2024-11-01T16:54:49.0972699Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-11-01T16:54:49.0973293Z  --security-opt seccomp=unconfined \ 2024-11-01T16:54:49.0973777Z  --cap-add=SYS_PTRACE \ 2024-11-01T16:54:49.0974197Z  --ipc=host \ 2024-11-01T16:54:49.0974580Z  --shm-size="${SHM_SIZE}" \ 2024-11-01T16:54:49.0975007Z  --tty \ 2024-11-01T16:54:49.0975343Z  --detach \ 2024-11-01T16:54:49.0975722Z  --name="${container_name}" \ 2024-11-01T16:54:49.0976150Z  --user jenkins \ 2024-11-01T16:54:49.0976668Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-11-01T16:54:49.0977270Z  -w /var/lib/jenkins/workspace \ 2024-11-01T16:54:49.0977740Z  "${DOCKER_IMAGE}" 2024-11-01T16:54:49.0978118Z ) 2024-11-01T16:54:49.0978535Z # Propagate download.pytorch.org IP to container 2024-11-01T16:54:49.0979540Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-11-01T16:54:49.0980603Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-11-01T16:54:49.0981607Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-11-01T16:54:49.0986790Z shell: /usr/bin/bash -e {0} 2024-11-01T16:54:49.0987166Z env: 2024-11-01T16:54:49.0987462Z GIT_DEFAULT_BRANCH: main 2024-11-01T16:54:49.0987922Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:54:49.0988577Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm75 2024-11-01T16:54:49.0989136Z PR_NUMBER: 138766 2024-11-01T16:54:49.0989499Z GITHUB_REPOSITORY: pytorch/pytorch 2024-11-01T16:54:49.0989940Z GITHUB_WORKFLOW: pull 2024-11-01T16:54:49.0990303Z GITHUB_JOB: test 2024-11-01T16:54:49.0990619Z GITHUB_RUN_ID: 11632514903 2024-11-01T16:54:49.0991001Z GITHUB_RUN_NUMBER: 261361 2024-11-01T16:54:49.0991380Z GITHUB_RUN_ATTEMPT: 1 2024-11-01T16:54:49.0991728Z JOB_ID: 32397131891 2024-11-01T16:54:49.0992428Z JOB_NAME: cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) 2024-11-01T16:54:49.0993282Z BRANCH: pull/138766 2024-11-01T16:54:49.0993690Z SHA1: d1aa4ef5a6c0ee2d57fb0086f5557e34537f0fea 2024-11-01T16:54:49.0994262Z BASE_SHA: 2055d3c8dc8b93c52eda6b9c07d716448d9e28c9 2024-11-01T16:54:49.0994790Z TEST_CONFIG: pr_time_benchmarks 2024-11-01T16:54:49.0995197Z SHARD_NUMBER: 1 2024-11-01T16:54:49.0995522Z NUM_TEST_SHARDS: 1 2024-11-01T16:54:49.0995842Z REENABLED_ISSUES: 2024-11-01T16:54:49.0996195Z CONTINUE_THROUGH_ERROR: False 2024-11-01T16:54:49.0996608Z VERBOSE_TEST_LOGS: False 2024-11-01T16:54:49.0996982Z TEST_SHOWLOCALS: False 2024-11-01T16:54:49.0997349Z NO_TEST_TIMEOUT: False 2024-11-01T16:54:49.0997686Z NO_TD: False 2024-11-01T16:54:49.0998004Z TD_DISTRIBUTED: False 2024-11-01T16:54:49.0998449Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-11-01T16:54:49.0998969Z SCCACHE_REGION: us-east-1 2024-11-01T16:54:49.0999359Z SCCACHE_S3_KEY_PREFIX: pull 2024-11-01T16:54:49.0999728Z SHM_SIZE: 2g 2024-11-01T16:54:49.1000944Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:54:49.1002244Z XLA_CUDA: 2024-11-01T16:54:49.1002770Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-11-01T16:54:49.1003435Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-11-01T16:54:49.1003905Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-11-01T16:54:49.1004320Z DASHBOARD_TAG: 2024-11-01T16:54:49.1004872Z HUGGING_FACE_HUB_TOKEN: *** 2024-11-01T16:54:49.1005461Z SCRIBE_GRAPHQL_ACCESS_TOKEN: *** 2024-11-01T16:54:49.1005879Z IS_A100_RUNNER: 0 2024-11-01T16:54:49.1006630Z ARTIFACTS_FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891 2024-11-01T16:54:49.1007392Z ##[endgroup] 2024-11-01T16:54:49.1199710Z Traceback (most recent call last): 2024-11-01T16:54:49.1202967Z File "/home/ec2-user/actions-runner/_work/pytorch/pytorch/.github/scripts/get_aws_session_tokens.py", line 2, in 2024-11-01T16:54:49.1205727Z import boto3 # type: ignore[import] 2024-11-01T16:54:49.1207262Z ModuleNotFoundError: No module named 'boto3' 2024-11-01T16:54:49.1227845Z + [[ pr_time_benchmarks == \m\u\l\t\i\g\p\u ]] 2024-11-01T16:54:49.1229436Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *onnx* ]] 2024-11-01T16:54:49.1229983Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-11-01T16:54:49.1236444Z +++ nproc --ignore=2 2024-11-01T16:54:49.1253410Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -e AWS_SESSION_TOKEN -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=94 -e SCCACHE_BUCKET -e SCCACHE_REGION -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG -e IS_A100_RUNNER -e ARTIFACTS_FILE_SUFFIX --env-file=/tmp/github_env_11632514903 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T16:55:00.1569003Z + container_name=b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T16:55:00.1575474Z + grep download.pytorch.org /etc/hosts 2024-11-01T16:55:00.1578037Z + docker exec -i b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 sudo bash -c '/bin/cat >> /etc/hosts' 2024-11-01T16:55:00.3232520Z + echo DOCKER_CONTAINER_ID=b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T16:55:00.3236317Z ++ echo dist/torch-2.6.0a0+gitd1aa4ef-cp310-cp310-linux_x86_64.whl 2024-11-01T16:55:00.3238301Z + docker exec -t b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 sh -c 'pip install dist/torch-2.6.0a0+gitd1aa4ef-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-11-01T16:55:00.6936347Z Processing ./dist/torch-2.6.0a0+gitd1aa4ef-cp310-cp310-linux_x86_64.whl (from torch==2.6.0a0+gitd1aa4ef) 2024-11-01T16:55:01.5242815Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (3.16.1) 2024-11-01T16:55:01.5245234Z Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (4.12.2) 2024-11-01T16:55:01.5247596Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (2.8.8) 2024-11-01T16:55:01.5249780Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (3.1.4) 2024-11-01T16:55:01.5251975Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (2024.10.0) 2024-11-01T16:55:01.5259733Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (1.13.1) 2024-11-01T16:55:01.5279677Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (1.3.0) 2024-11-01T16:55:01.5293574Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (3.3.0) 2024-11-01T16:55:01.5313659Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (1.22.4) 2024-11-01T16:55:01.5689963Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.6.0a0+gitd1aa4ef->torch==2.6.0a0+gitd1aa4ef) (3.0.2) 2024-11-01T16:55:02.1335802Z Installing collected packages: torch 2024-11-01T16:55:12.0715439Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-11-01T16:55:12.0717151Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-11-01T16:55:12.0718022Z Successfully installed torch-2.6.0a0+gitd1aa4ef 2024-11-01T16:55:12.1442962Z + export TERM=vt100 2024-11-01T16:55:12.1443376Z + TERM=vt100 2024-11-01T16:55:12.1443739Z ++ dirname .ci/pytorch/test.sh 2024-11-01T16:55:12.1449588Z + source .ci/pytorch/common.sh 2024-11-01T16:55:12.1450748Z +++ dirname .ci/pytorch/common.sh 2024-11-01T16:55:12.1457076Z ++ source .ci/pytorch/common_utils.sh 2024-11-01T16:55:12.1458756Z +++ declare -f -t trap_add 2024-11-01T16:55:12.1464836Z ++ set -ex 2024-11-01T16:55:12.1465373Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *rocm* ]] 2024-11-01T16:55:12.1465949Z ++ BUILD_TEST_LIBTORCH=0 2024-11-01T16:55:12.1466580Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 != *rocm* ]] 2024-11-01T16:55:12.1471292Z ++ stat -c %u /var/lib/jenkins/workspace 2024-11-01T16:55:12.1489951Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-11-01T16:55:12.1491030Z + trap_add cleanup_workspace EXIT 2024-11-01T16:55:12.1491792Z + trap_add_cmd=cleanup_workspace 2024-11-01T16:55:12.1492194Z + shift 2024-11-01T16:55:12.1495140Z + for trap_add_name in "$@" 2024-11-01T16:55:12.1495690Z +++ trap -p EXIT 2024-11-01T16:55:12.1496087Z ++ eval 'extract_trap_cmd ' 2024-11-01T16:55:12.1496470Z +++ extract_trap_cmd 2024-11-01T16:55:12.1496858Z +++ printf '%s\n' '' 2024-11-01T16:55:12.1497287Z ++ printf '%s\n' cleanup_workspace 2024-11-01T16:55:12.1497755Z + trap -- ' 2024-11-01T16:55:12.1498108Z cleanup_workspace' EXIT 2024-11-01T16:55:12.1498629Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-11-01T16:55:12.5475233Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-11-01T16:55:12.5498445Z + echo 'Environment variables:' 2024-11-01T16:55:12.5499632Z Environment variables: 2024-11-01T16:55:12.5500528Z + env 2024-11-01T16:55:12.5507963Z INSTALLED_DB=yes 2024-11-01T16:55:12.5508538Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-11-01T16:55:12.5508981Z NVIDIA_VISIBLE_DEVICES=all 2024-11-01T16:55:12.5509414Z NV_NVML_DEV_VERSION=12.1.105-1 2024-11-01T16:55:12.5510178Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-11-01T16:55:12.5510835Z CONTINUE_THROUGH_ERROR=False 2024-11-01T16:55:12.5511391Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-11-01T16:55:12.5511992Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-11-01T16:55:12.5512687Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm75 2024-11-01T16:55:12.5513263Z HOSTNAME=b27d0eddff79 2024-11-01T16:55:12.5514275Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5515242Z GITHUB_ACTION=__self 2024-11-01T16:55:12.5515604Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-11-01T16:55:12.5521804Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-11-01T16:55:12.5526765Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-11-01T16:55:12.5527342Z NV_NVTX_VERSION=12.1.105-1 2024-11-01T16:55:12.5527735Z GITHUB_RUN_NUMBER=261361 2024-11-01T16:55:12.5528116Z TEST_CONFIG=pr_time_benchmarks 2024-11-01T16:55:12.5528545Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-11-01T16:55:12.5529071Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-11-01T16:55:12.5529521Z IS_A100_RUNNER=0 2024-11-01T16:55:12.5529976Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-11-01T16:55:12.5530469Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-11-01T16:55:12.5531297Z SCRIBE_GRAPHQL_ACCESS_TOKEN=*** 2024-11-01T16:55:12.5531860Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-11-01T16:55:12.5532291Z GITHUB_TRIGGERING_ACTOR=c00w 2024-11-01T16:55:12.5532784Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-11-01T16:55:12.5533329Z GITHUB_REF_TYPE=branch 2024-11-01T16:55:12.5533711Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-11-01T16:55:12.5534100Z NCCL_VERSION=2.17.1-1 2024-11-01T16:55:12.5534535Z BASE_SHA=2055d3c8dc8b93c52eda6b9c07d716448d9e28c9 2024-11-01T16:55:12.5535038Z XLA_CUDA= 2024-11-01T16:55:12.5535502Z HUGGING_FACE_HUB_TOKEN=*** 2024-11-01T16:55:12.5538191Z *** 2024-11-01T16:55:12.5538515Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-11-01T16:55:12.5538951Z GITHUB_REPOSITORY_ID=65600975 2024-11-01T16:55:12.5539334Z GITHUB_ACTIONS=true 2024-11-01T16:55:12.5539692Z NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:55:12.5540262Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-11-01T16:55:12.5540869Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-11-01T16:55:12.5541548Z SHA1=d1aa4ef5a6c0ee2d57fb0086f5557e34537f0fea 2024-11-01T16:55:12.5542120Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-11-01T16:55:12.5542656Z GITHUB_SHA=368da7510f5d755034681777a4e5ae6d33c07b38 2024-11-01T16:55:12.5543472Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/138766/merge 2024-11-01T16:55:12.5544200Z UCC_HOME=/usr 2024-11-01T16:55:12.5544558Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-11-01T16:55:12.5544981Z VERBOSE_TEST_LOGS=False 2024-11-01T16:55:12.5545348Z NVIDIA_PRODUCT_NAME=CUDA 2024-11-01T16:55:12.5545860Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-11-01T16:55:12.5546392Z GITHUB_REF=refs/pull/138766/merge 2024-11-01T16:55:12.5546857Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-11-01T16:55:12.5547257Z SHARD_NUMBER=1 2024-11-01T16:55:12.5547590Z GITHUB_REF_PROTECTED=false 2024-11-01T16:55:12.5547975Z HOME=/var/lib/jenkins 2024-11-01T16:55:12.5548373Z GITHUB_API_URL=https://api.github.com 2024-11-01T16:55:12.5548873Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-11-01T16:55:12.5549406Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-11-01T16:55:12.5549933Z SCCACHE_S3_KEY_PREFIX=pull 2024-11-01T16:55:12.5550318Z CUDA_VERSION=12.1.1 2024-11-01T16:55:12.5550787Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-11-01T16:55:12.5551279Z NUM_TEST_SHARDS=1 2024-11-01T16:55:12.5551602Z UCX_HOME=/usr 2024-11-01T16:55:12.5552175Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-11-01T16:55:12.5553472Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5555008Z JOB_NAME=cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) 2024-11-01T16:55:12.5556464Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5557804Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-11-01T16:55:12.5558681Z GITHUB_EVENT_NAME=pull_request 2024-11-01T16:55:12.5559061Z DASHBOARD_TAG= 2024-11-01T16:55:12.5559378Z GITHUB_RUN_ID=11632514903 2024-11-01T16:55:12.5559875Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-11-01T16:55:12.5560468Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-11-01T16:55:12.5561632Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5562623Z GITHUB_ACTOR=c00w 2024-11-01T16:55:12.5562976Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-11-01T16:55:12.5563383Z PR_NUMBER=138766 2024-11-01T16:55:12.5563707Z GITHUB_RUN_ATTEMPT=1 2024-11-01T16:55:12.5564068Z ANACONDA_PYTHON_VERSION=3.10 2024-11-01T16:55:12.5564548Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-11-01T16:55:12.5565036Z TERM=vt100 2024-11-01T16:55:12.5565391Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-11-01T16:55:12.5565842Z INSTALLED_VISION=yes 2024-11-01T16:55:12.5566185Z BRANCH=pull/138766 2024-11-01T16:55:12.5566563Z SCCACHE_REGION=us-east-1 2024-11-01T16:55:12.5566930Z OPENSSL_ROOT_DIR=/opt/openssl 2024-11-01T16:55:12.5567359Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-11-01T16:55:12.5567822Z CUDA_PATH=/usr/local/cuda 2024-11-01T16:55:12.5568679Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-11-01T16:55:12.5569535Z GITHUB_SERVER_URL=https://github.com 2024-11-01T16:55:12.5570048Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-11-01T16:55:12.5570557Z REENABLED_ISSUES= 2024-11-01T16:55:12.5570862Z SHLVL=1 2024-11-01T16:55:12.5571127Z MAX_JOBS=94 2024-11-01T16:55:12.5571452Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-11-01T16:55:12.5571815Z NVARCH=x86_64 2024-11-01T16:55:12.5572114Z GITHUB_ACTOR_ID=486199 2024-11-01T16:55:12.5572586Z GITHUB_WORKFLOW_SHA=368da7510f5d755034681777a4e5ae6d33c07b38 2024-11-01T16:55:12.5573140Z GITHUB_REF_NAME=138766/merge 2024-11-01T16:55:12.5573603Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-11-01T16:55:12.5574404Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-11-01T16:55:12.5575043Z GITHUB_JOB=test 2024-11-01T16:55:12.5575470Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-11-01T16:55:12.5576078Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-11-01T16:55:12.5576639Z NO_TEST_TIMEOUT=False 2024-11-01T16:55:12.5576981Z TD_DISTRIBUTED=False 2024-11-01T16:55:12.5577363Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-11-01T16:55:12.5577831Z GITHUB_REPOSITORY=pytorch/pytorch 2024-11-01T16:55:12.5578273Z NV_NVPROF_VERSION=12.1.105-1 2024-11-01T16:55:12.5578658Z GITHUB_RETENTION_DAYS=90 2024-11-01T16:55:12.5579019Z OPENSSL_DIR=/opt/openssl 2024-11-01T16:55:12.5579375Z GITHUB_ACTION_REPOSITORY= 2024-11-01T16:55:12.5580556Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-11-01T16:55:12.5581804Z GITHUB_BASE_REF=gh/c00w/2/base 2024-11-01T16:55:12.5582632Z ARTIFACTS_FILE_SUFFIX=test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891 2024-11-01T16:55:12.5583418Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-11-01T16:55:12.5583805Z CI=true 2024-11-01T16:55:12.5584119Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-11-01T16:55:12.5584556Z GITHUB_REPOSITORY_OWNER=pytorch 2024-11-01T16:55:12.5584952Z JOB_ID=32397131891 2024-11-01T16:55:12.5585274Z INSTALLED_PROTOBUF=yes 2024-11-01T16:55:12.5585631Z GITHUB_HEAD_REF=gh/c00w/2/head 2024-11-01T16:55:12.5586002Z GITHUB_ACTION_REF= 2024-11-01T16:55:12.5586471Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-11-01T16:55:12.5586980Z TEST_SHOWLOCALS=False 2024-11-01T16:55:12.5587389Z GITHUB_WORKFLOW=pull 2024-11-01T16:55:12.5587750Z DEBIAN_FRONTEND=noninteractive 2024-11-01T16:55:12.5588788Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5589726Z NO_TD=False 2024-11-01T16:55:12.5590042Z SKIP_SCCACHE_INITIALIZATION=1 2024-11-01T16:55:12.5590428Z _=/usr/bin/env 2024-11-01T16:55:12.5590947Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-11-01T16:55:12.5743655Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-11-01T16:55:12.5744732Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-11-01T16:55:12.5745767Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-11-01T16:55:12.5746803Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-11-01T16:55:12.5747506Z + BUILD_DIR=build 2024-11-01T16:55:12.5747861Z + BUILD_RENAMED_DIR=build_renamed 2024-11-01T16:55:12.5748294Z + BUILD_BIN_DIR=build/bin 2024-11-01T16:55:12.5748646Z + SHARD_NUMBER=1 2024-11-01T16:55:12.5748965Z + NUM_TEST_SHARDS=1 2024-11-01T16:55:12.5749308Z + export VALGRIND=ON 2024-11-01T16:55:12.5749649Z + VALGRIND=ON 2024-11-01T16:55:12.5750172Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *clang9* ]] 2024-11-01T16:55:12.5750721Z + [[ 0 == \1 ]] 2024-11-01T16:55:12.5751035Z + [[ False == \1 ]] 2024-11-01T16:55:12.5751570Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 != *bazel* ]] 2024-11-01T16:55:12.5752170Z ++ realpath build/custom_test_artifacts 2024-11-01T16:55:12.5753111Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-11-01T16:55:12.5753875Z + [[ -n '' ]] 2024-11-01T16:55:12.5754265Z + echo 'Environment variables' 2024-11-01T16:55:12.5754684Z Environment variables 2024-11-01T16:55:12.5755024Z + env 2024-11-01T16:55:12.5759844Z INSTALLED_DB=yes 2024-11-01T16:55:12.5760225Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-11-01T16:55:12.5760661Z NVIDIA_VISIBLE_DEVICES=all 2024-11-01T16:55:12.5761086Z NV_NVML_DEV_VERSION=12.1.105-1 2024-11-01T16:55:12.5761764Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-11-01T16:55:12.5762421Z CONTINUE_THROUGH_ERROR=False 2024-11-01T16:55:12.5762975Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-11-01T16:55:12.5763734Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-11-01T16:55:12.5764369Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm75 2024-11-01T16:55:12.5764938Z HOSTNAME=b27d0eddff79 2024-11-01T16:55:12.5765935Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5766891Z GITHUB_ACTION=__self 2024-11-01T16:55:12.5767270Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-11-01T16:55:12.5771976Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-11-01T16:55:12.5776776Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-11-01T16:55:12.5777348Z NV_NVTX_VERSION=12.1.105-1 2024-11-01T16:55:12.5777725Z GITHUB_RUN_NUMBER=261361 2024-11-01T16:55:12.5778117Z TEST_CONFIG=pr_time_benchmarks 2024-11-01T16:55:12.5778545Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-11-01T16:55:12.5779070Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-11-01T16:55:12.5779541Z IS_A100_RUNNER=0 2024-11-01T16:55:12.5780043Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-11-01T16:55:12.5780570Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-11-01T16:55:12.5781368Z SCRIBE_GRAPHQL_ACCESS_TOKEN=*** 2024-11-01T16:55:12.5781859Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-11-01T16:55:12.5782298Z GITHUB_TRIGGERING_ACTOR=c00w 2024-11-01T16:55:12.5782812Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-11-01T16:55:12.5783340Z GITHUB_REF_TYPE=branch 2024-11-01T16:55:12.5783796Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-11-01T16:55:12.5784225Z NCCL_VERSION=2.17.1-1 2024-11-01T16:55:12.5784678Z BASE_SHA=2055d3c8dc8b93c52eda6b9c07d716448d9e28c9 2024-11-01T16:55:12.5785194Z XLA_CUDA= 2024-11-01T16:55:12.5785735Z HUGGING_FACE_HUB_TOKEN=*** 2024-11-01T16:55:12.5786183Z *** 2024-11-01T16:55:12.5786507Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-11-01T16:55:12.5786942Z GITHUB_REPOSITORY_ID=65600975 2024-11-01T16:55:12.5787366Z GITHUB_ACTIONS=true 2024-11-01T16:55:12.5787752Z NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T16:55:12.5788341Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-11-01T16:55:12.5788966Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-11-01T16:55:12.5789498Z SHA1=d1aa4ef5a6c0ee2d57fb0086f5557e34537f0fea 2024-11-01T16:55:12.5790101Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-11-01T16:55:12.5790660Z GITHUB_SHA=368da7510f5d755034681777a4e5ae6d33c07b38 2024-11-01T16:55:12.5791496Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/138766/merge 2024-11-01T16:55:12.5792245Z UCC_HOME=/usr 2024-11-01T16:55:12.5792601Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-11-01T16:55:12.5793060Z VERBOSE_TEST_LOGS=False 2024-11-01T16:55:12.5793456Z NVIDIA_PRODUCT_NAME=CUDA 2024-11-01T16:55:12.5793985Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-11-01T16:55:12.5794532Z GITHUB_REF=refs/pull/138766/merge 2024-11-01T16:55:12.5794996Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-11-01T16:55:12.5795428Z SHARD_NUMBER=1 2024-11-01T16:55:12.5795777Z GITHUB_REF_PROTECTED=false 2024-11-01T16:55:12.5796177Z HOME=/var/lib/jenkins 2024-11-01T16:55:12.5796603Z GITHUB_API_URL=https://api.github.com 2024-11-01T16:55:12.5797112Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-11-01T16:55:12.5797644Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-11-01T16:55:12.5798202Z SCCACHE_S3_KEY_PREFIX=pull 2024-11-01T16:55:12.5798602Z CUDA_VERSION=12.1.1 2024-11-01T16:55:12.5799156Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-11-01T16:55:12.5799686Z NUM_TEST_SHARDS=1 2024-11-01T16:55:12.5800011Z UCX_HOME=/usr 2024-11-01T16:55:12.5800623Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-11-01T16:55:12.5801948Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5803418Z JOB_NAME=cuda12.1-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) 2024-11-01T16:55:12.5804892Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5806291Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-11-01T16:55:12.5807074Z GITHUB_EVENT_NAME=pull_request 2024-11-01T16:55:12.5807502Z DASHBOARD_TAG= 2024-11-01T16:55:12.5807855Z GITHUB_RUN_ID=11632514903 2024-11-01T16:55:12.5808391Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-11-01T16:55:12.5809028Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-11-01T16:55:12.5810301Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5811350Z GITHUB_ACTOR=c00w 2024-11-01T16:55:12.5811753Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-11-01T16:55:12.5812198Z PR_NUMBER=138766 2024-11-01T16:55:12.5830407Z GITHUB_RUN_ATTEMPT=1 2024-11-01T16:55:12.5830873Z VALGRIND=ON 2024-11-01T16:55:12.5831209Z ANACONDA_PYTHON_VERSION=3.10 2024-11-01T16:55:12.5831719Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-11-01T16:55:12.5832215Z TERM=vt100 2024-11-01T16:55:12.5832779Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-11-01T16:55:12.5833247Z INSTALLED_VISION=yes 2024-11-01T16:55:12.5833598Z BRANCH=pull/138766 2024-11-01T16:55:12.5833985Z SCCACHE_REGION=us-east-1 2024-11-01T16:55:12.5834355Z OPENSSL_ROOT_DIR=/opt/openssl 2024-11-01T16:55:12.5834783Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-11-01T16:55:12.5835260Z CUDA_PATH=/usr/local/cuda 2024-11-01T16:55:12.5836144Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-11-01T16:55:12.5837027Z GITHUB_SERVER_URL=https://github.com 2024-11-01T16:55:12.5837563Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-11-01T16:55:12.5838068Z REENABLED_ISSUES= 2024-11-01T16:55:12.5838377Z SHLVL=1 2024-11-01T16:55:12.5838643Z MAX_JOBS=94 2024-11-01T16:55:12.5838972Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-11-01T16:55:12.5839351Z NVARCH=x86_64 2024-11-01T16:55:12.5839648Z GITHUB_ACTOR_ID=486199 2024-11-01T16:55:12.5840126Z GITHUB_WORKFLOW_SHA=368da7510f5d755034681777a4e5ae6d33c07b38 2024-11-01T16:55:12.5840692Z GITHUB_REF_NAME=138766/merge 2024-11-01T16:55:12.5841170Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-11-01T16:55:12.5841928Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-11-01T16:55:12.5842568Z GITHUB_JOB=test 2024-11-01T16:55:12.5843005Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-11-01T16:55:12.5843632Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-11-01T16:55:12.5844206Z NO_TEST_TIMEOUT=False 2024-11-01T16:55:12.5844552Z TD_DISTRIBUTED=False 2024-11-01T16:55:12.5844947Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-11-01T16:55:12.5845427Z GITHUB_REPOSITORY=pytorch/pytorch 2024-11-01T16:55:12.5845881Z NV_NVPROF_VERSION=12.1.105-1 2024-11-01T16:55:12.5846274Z GITHUB_RETENTION_DAYS=90 2024-11-01T16:55:12.5846646Z OPENSSL_DIR=/opt/openssl 2024-11-01T16:55:12.5847009Z GITHUB_ACTION_REPOSITORY= 2024-11-01T16:55:12.5848231Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-11-01T16:55:12.5849518Z GITHUB_BASE_REF=gh/c00w/2/base 2024-11-01T16:55:12.5850370Z ARTIFACTS_FILE_SUFFIX=test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891 2024-11-01T16:55:12.5851180Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-11-01T16:55:12.5851694Z CI=true 2024-11-01T16:55:12.5852021Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-11-01T16:55:12.5852478Z GITHUB_REPOSITORY_OWNER=pytorch 2024-11-01T16:55:12.5852887Z JOB_ID=32397131891 2024-11-01T16:55:12.5853223Z INSTALLED_PROTOBUF=yes 2024-11-01T16:55:12.5853589Z GITHUB_HEAD_REF=gh/c00w/2/head 2024-11-01T16:55:12.5853971Z GITHUB_ACTION_REF= 2024-11-01T16:55:12.5854451Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-11-01T16:55:12.5854973Z TEST_SHOWLOCALS=False 2024-11-01T16:55:12.5855327Z GITHUB_WORKFLOW=pull 2024-11-01T16:55:12.5855692Z DEBIAN_FRONTEND=noninteractive 2024-11-01T16:55:12.5856760Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_f1a19e82-2abd-4f1c-a500-a1ed720adcd0 2024-11-01T16:55:12.5857729Z NO_TD=False 2024-11-01T16:55:12.5858050Z SKIP_SCCACHE_INITIALIZATION=1 2024-11-01T16:55:12.5858442Z _=/usr/bin/env 2024-11-01T16:55:12.5858804Z + echo 'Testing pytorch' 2024-11-01T16:55:12.5859175Z Testing pytorch 2024-11-01T16:55:12.5859506Z + export LANG=C.UTF-8 2024-11-01T16:55:12.5859870Z + LANG=C.UTF-8 2024-11-01T16:55:12.5860181Z + PR_NUMBER=138766 2024-11-01T16:55:12.5860564Z + [[ pr_time_benchmarks == \d\e\f\a\u\l\t ]] 2024-11-01T16:55:12.5861119Z + [[ pr_time_benchmarks == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-11-01T16:55:12.5861646Z + [[ pr_time_benchmarks == \s\l\o\w ]] 2024-11-01T16:55:12.5862352Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *slow-gradcheck* ]] 2024-11-01T16:55:12.5863145Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *cuda* ]] 2024-11-01T16:55:12.5863758Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-11-01T16:55:12.5864278Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-11-01T16:55:12.5864751Z + [[ pr_time_benchmarks == *crossref* ]] 2024-11-01T16:55:12.5865455Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *rocm* ]] 2024-11-01T16:55:12.5866188Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *xpu* ]] 2024-11-01T16:55:12.5866936Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 != *-bazel-* ]] 2024-11-01T16:55:12.5867571Z + pip_install --user ninja==1.10.2 2024-11-01T16:55:12.5868176Z + pip install --progress-bar off --user ninja==1.10.2 2024-11-01T16:55:15.6618047Z Collecting ninja==1.10.2 2024-11-01T16:55:15.6820769Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-11-01T16:55:15.6970862Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-11-01T16:55:16.2418774Z Installing collected packages: ninja 2024-11-01T16:55:16.2492403Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-11-01T16:55:16.2493940Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-11-01T16:55:16.2581854Z Successfully installed ninja-1.10.2 2024-11-01T16:55:16.3271408Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-11-01T16:55:16.3274590Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-11-01T16:55:16.3276318Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *aarch64* ]] 2024-11-01T16:55:16.3276890Z + install_tlparse 2024-11-01T16:55:16.3277295Z + pip_install --user tlparse==0.3.25 2024-11-01T16:55:16.3277928Z + pip install --progress-bar off --user tlparse==0.3.25 2024-11-01T16:55:16.7008092Z Collecting tlparse==0.3.25 2024-11-01T16:55:16.7173610Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-11-01T16:55:16.7289833Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-11-01T16:55:17.3020919Z Installing collected packages: tlparse 2024-11-01T16:55:17.3389583Z Successfully installed tlparse-0.3.25 2024-11-01T16:55:17.4102150Z ++ python -m site --user-base 2024-11-01T16:55:17.4341998Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-11-01T16:55:17.4343933Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *asan* ]] 2024-11-01T16:55:17.4344708Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 == *-debug* ]] 2024-11-01T16:55:17.4345474Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm75 != *-bazel-* ]] 2024-11-01T16:55:17.4346581Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm75. Expect the assertion to pass' 2024-11-01T16:55:17.4347933Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm75. Expect the assertion to pass 2024-11-01T16:55:17.4348738Z + cd test 2024-11-01T16:55:17.4349329Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-11-01T16:55:19.0305012Z + [[ pr_time_benchmarks == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-11-01T16:55:19.0305727Z + [[ pr_time_benchmarks == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-11-01T16:55:19.0307046Z + DYNAMO_BENCHMARK_FLAGS=() 2024-11-01T16:55:19.0307514Z + [[ pr_time_benchmarks == *pr_time_benchmarks* ]] 2024-11-01T16:55:19.0308034Z + pr_time_benchmarks 2024-11-01T16:55:19.0308649Z + pip_install --user fbscribelogger 2024-11-01T16:55:19.0309279Z + pip install --progress-bar off --user fbscribelogger 2024-11-01T16:55:19.3546129Z Requirement already satisfied: fbscribelogger in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (0.1.6) 2024-11-01T16:55:19.3561720Z Requirement already satisfied: aiohttp<4.0.0,>=3.10.4 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbscribelogger) (3.10.10) 2024-11-01T16:55:19.3563907Z Requirement already satisfied: thriftpy2<0.6.0,>=0.5.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbscribelogger) (0.5.2) 2024-11-01T16:55:19.3566886Z Requirement already satisfied: typing-extensions<5.0.0,>=4.12.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbscribelogger) (4.12.2) 2024-11-01T16:55:19.3599264Z Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (2.4.3) 2024-11-01T16:55:19.3601454Z Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (1.3.1) 2024-11-01T16:55:19.3603556Z Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (24.2.0) 2024-11-01T16:55:19.3607236Z Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (1.5.0) 2024-11-01T16:55:19.3612007Z Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (6.1.0) 2024-11-01T16:55:19.3616396Z Requirement already satisfied: yarl<2.0,>=1.12.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (1.17.0) 2024-11-01T16:55:19.3623165Z Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (4.0.3) 2024-11-01T16:55:19.3642986Z Requirement already satisfied: Cython>=3.0.10 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from thriftpy2<0.6.0,>=0.5.2->fbscribelogger) (3.0.11) 2024-11-01T16:55:19.3645091Z Requirement already satisfied: ply<4.0,>=3.4 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from thriftpy2<0.6.0,>=0.5.2->fbscribelogger) (3.11) 2024-11-01T16:55:19.3647449Z Requirement already satisfied: six~=1.15 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from thriftpy2<0.6.0,>=0.5.2->fbscribelogger) (1.16.0) 2024-11-01T16:55:19.3840923Z Requirement already satisfied: idna>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from yarl<2.0,>=1.12.0->aiohttp<4.0.0,>=3.10.4->fbscribelogger) (3.10) 2024-11-01T16:55:19.3843567Z Requirement already satisfied: propcache>=0.2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from yarl<2.0,>=1.12.0->aiohttp<4.0.0,>=3.10.4->fbscribelogger) (0.2.0) 2024-11-01T16:55:19.9599713Z ++ pwd 2024-11-01T16:55:19.9600667Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-11-01T16:55:19.9601472Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-11-01T16:55:19.9626839Z ++ pwd 2024-11-01T16:55:19.9629453Z + PYTHONPATH=/var/lib/jenkins/workspace/benchmarks/dynamo/pr_time_benchmarks 2024-11-01T16:55:19.9632049Z + source benchmarks/dynamo/pr_time_benchmarks/benchmark_runner.sh /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv benchmarks/dynamo/pr_time_benchmarks/benchmarks 2024-11-01T16:55:19.9633580Z ++ '[' 2 -eq 0 ']' 2024-11-01T16:55:19.9633957Z ++ '[' 2 -eq 1 ']' 2024-11-01T16:55:19.9634779Z ++ output_file=/var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv 2024-11-01T16:55:19.9635829Z ++ python_programs_dir=benchmarks/dynamo/pr_time_benchmarks/benchmarks 2024-11-01T16:55:19.9636502Z +++ date +%s 2024-11-01T16:55:19.9640800Z ++ start=1730480119 2024-11-01T16:55:19.9641658Z ++ for file in $python_programs_dir/*.py 2024-11-01T16:55:19.9643066Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/add_loop.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv 2024-11-01T16:55:22.3455118Z collecting compile time instruction count for add_loop_eager 2024-11-01T16:55:24.4456567Z compile time instruction count for iteration 0 is 3137333901 2024-11-01T16:55:25.3339729Z compile time instruction count for iteration 1 is 3027725352 2024-11-01T16:55:26.2134209Z compile time instruction count for iteration 2 is 3025865692 2024-11-01T16:55:27.0894410Z compile time instruction count for iteration 3 is 3022834455 2024-11-01T16:55:27.9702709Z compile time instruction count for iteration 4 is 3023102926 2024-11-01T16:55:28.0017350Z collecting compile time instruction count for add_loop_eager_dynamic 2024-11-01T16:55:29.5414938Z compile time instruction count for iteration 0 is 5828718707 2024-11-01T16:55:31.0453849Z compile time instruction count for iteration 1 is 5606586563 2024-11-01T16:55:32.5401885Z compile time instruction count for iteration 2 is 5604716603 2024-11-01T16:55:34.0388304Z compile time instruction count for iteration 3 is 5603253484 2024-11-01T16:55:35.5388549Z compile time instruction count for iteration 4 is 5600346214 2024-11-01T16:55:35.5434367Z collecting compile time instruction count for add_loop_inductor 2024-11-01T16:55:52.6876373Z compile time instruction count for iteration 0 is 46379642710 2024-11-01T16:56:01.2176901Z compile time instruction count for iteration 1 is 24386372422 2024-11-01T16:56:09.8277104Z compile time instruction count for iteration 2 is 24378418082 2024-11-01T16:56:18.4840889Z compile time instruction count for iteration 3 is 24385694637 2024-11-01T16:56:27.1826108Z compile time instruction count for iteration 4 is 24395306049 2024-11-01T16:56:27.2450573Z collecting compile time instruction count for add_loop_inductor_gpu 2024-11-01T16:56:56.8775062Z compile time instruction count for iteration 0 is 23219644178 2024-11-01T16:57:26.2725520Z compile time instruction count for iteration 1 is 23083851067 2024-11-01T16:57:55.7276757Z compile time instruction count for iteration 2 is 23084211347 2024-11-01T16:58:25.1847739Z compile time instruction count for iteration 3 is 23084854983 2024-11-01T16:58:55.7866797Z compile time instruction count for iteration 4 is 23094885199 2024-11-01T16:58:55.8399034Z collecting compile time instruction count for add_loop_inductor_dynamic_gpu 2024-11-01T16:59:29.5249689Z compile time instruction count for iteration 0 is 40627145777 2024-11-01T17:00:03.4853131Z compile time instruction count for iteration 1 is 40482165743 2024-11-01T17:00:37.5927945Z compile time instruction count for iteration 2 is 40479893433 2024-11-01T17:01:11.7114736Z compile time instruction count for iteration 3 is 40484378305 2024-11-01T17:01:45.9430168Z compile time instruction count for iteration 4 is 40475270434 2024-11-01T17:01:50.2547614Z ++ for file in $python_programs_dir/*.py 2024-11-01T17:01:50.2549289Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/aotdispatcher.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv 2024-11-01T17:01:52.5841796Z collecting compile time instruction count for aotdispatcher_inference_nosubclass_cpu 2024-11-01T17:01:54.9554428Z compile time instruction count for iteration 0 is 6054513366 2024-11-01T17:01:55.6320927Z compile time instruction count for iteration 1 is 2019333125 2024-11-01T17:01:56.3129332Z compile time instruction count for iteration 2 is 2004328764 2024-11-01T17:01:56.9803015Z compile time instruction count for iteration 3 is 1998066414 2024-11-01T17:01:57.6463594Z compile time instruction count for iteration 4 is 1996478859 2024-11-01T17:01:57.6789409Z collecting compile time instruction count for aotdispatcher_training_nosubclass_cpu 2024-11-01T17:01:58.7837630Z compile time instruction count for iteration 0 is 4018940603 2024-11-01T17:01:59.8252999Z compile time instruction count for iteration 1 is 3823223999 2024-11-01T17:02:00.9084472Z compile time instruction count for iteration 2 is 3820753868 2024-11-01T17:02:01.9495794Z compile time instruction count for iteration 3 is 3818136055 2024-11-01T17:02:03.0271163Z compile time instruction count for iteration 4 is 3818120096 2024-11-01T17:02:03.0365688Z collecting compile time instruction count for aotdispatcher_inference_subclass_cpu 2024-11-01T17:02:04.5646319Z compile time instruction count for iteration 0 is 5777671267 2024-11-01T17:02:06.0932798Z compile time instruction count for iteration 1 is 5774226835 2024-11-01T17:02:07.6347179Z compile time instruction count for iteration 2 is 5772792993 2024-11-01T17:02:09.1704138Z compile time instruction count for iteration 3 is 5774627171 2024-11-01T17:02:10.7036111Z compile time instruction count for iteration 4 is 5771607233 2024-11-01T17:02:10.7155537Z collecting compile time instruction count for aotdispatcher_training_subclass_cpu 2024-11-01T17:02:13.1667672Z compile time instruction count for iteration 0 is 10261886553 2024-11-01T17:02:15.6390401Z compile time instruction count for iteration 1 is 10260669039 2024-11-01T17:02:18.1122436Z compile time instruction count for iteration 2 is 10262351783 2024-11-01T17:02:20.5805557Z compile time instruction count for iteration 3 is 10258440935 2024-11-01T17:02:23.0469473Z compile time instruction count for iteration 4 is 10259769073 2024-11-01T17:02:23.8885404Z ++ for file in $python_programs_dir/*.py 2024-11-01T17:02:23.8889063Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/aotdispatcher_partitioner.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv 2024-11-01T17:02:26.2249588Z collecting compile time instruction count for aotdispatcher_partitioner_cpu 2024-11-01T17:02:30.3211925Z compile time instruction count for iteration 0 is 13322069677 2024-11-01T17:02:32.6515921Z compile time instruction count for iteration 1 is 9033963903 2024-11-01T17:02:34.9759038Z compile time instruction count for iteration 2 is 9021374933 2024-11-01T17:02:37.3006560Z compile time instruction count for iteration 3 is 9009572973 2024-11-01T17:02:39.6335753Z compile time instruction count for iteration 4 is 9008664830 2024-11-01T17:02:40.4829439Z ++ for file in $python_programs_dir/*.py 2024-11-01T17:02:40.4831225Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/basic_modules_benchmarks.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv 2024-11-01T17:02:42.8071430Z collecting compile time instruction count for basic_modules_ListOfLinears_eager 2024-11-01T17:02:44.3612845Z compile time instruction count for iteration 0 is 1183972370 2024-11-01T17:02:44.6998162Z compile time instruction count for iteration 1 is 1046083175 2024-11-01T17:02:45.0220794Z compile time instruction count for iteration 2 is 1040855065 2024-11-01T17:02:45.3487614Z compile time instruction count for iteration 3 is 1035371394 2024-11-01T17:02:45.6694965Z compile time instruction count for iteration 4 is 1030754764 2024-11-01T17:02:45.7012762Z collecting compile time instruction count for basic_modules_ListOfLinears_inductor 2024-11-01T17:03:01.1635169Z compile time instruction count for iteration 0 is 37971561602 2024-11-01T17:03:07.7080746Z compile time instruction count for iteration 1 is 19266468265 2024-11-01T17:03:14.2819074Z compile time instruction count for iteration 2 is 19233734076 2024-11-01T17:03:20.8752261Z compile time instruction count for iteration 3 is 19196438015 2024-11-01T17:03:27.4953253Z compile time instruction count for iteration 4 is 19198271390 2024-11-01T17:03:27.5916472Z collecting compile time instruction count for basic_modules_ListOfLinears_inductor_gpu 2024-11-01T17:03:30.9091908Z W1101 17:03:30.908000 1514 site-packages/torch/_inductor/utils.py:1095] [0/0] Not enough SMs to use max_autotune_gemm mode 2024-11-01T17:03:31.8778240Z compile time instruction count for iteration 0 is 17012031275 2024-11-01T17:03:36.0733223Z compile time instruction count for iteration 1 is 16889016925 2024-11-01T17:03:40.2735760Z compile time instruction count for iteration 2 is 16883431014 2024-11-01T17:03:44.4919791Z compile time instruction count for iteration 3 is 16866875596 2024-11-01T17:03:48.7176859Z compile time instruction count for iteration 4 is 16849596082 2024-11-01T17:03:48.7536957Z collecting compile time instruction count for basic_modules_ListOfLinears_inductor_gpu_force_shape_pad 2024-11-01T17:03:53.7007690Z compile time instruction count for iteration 0 is 15940774602 2024-11-01T17:03:58.7273488Z compile time instruction count for iteration 1 is 15814487232 2024-11-01T17:04:03.7909712Z compile time instruction count for iteration 2 is 15813889578 2024-11-01T17:04:08.8506848Z compile time instruction count for iteration 3 is 15814964270 2024-11-01T17:04:13.7134159Z compile time instruction count for iteration 4 is 15814919504 2024-11-01T17:04:15.6927102Z ++ for file in $python_programs_dir/*.py 2024-11-01T17:04:15.6928801Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/sum_floordiv.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv 2024-11-01T17:04:18.0161554Z collecting compile time instruction count for sum_floordiv_regression 2024-11-01T17:04:20.3381116Z compile time instruction count for iteration 0 is 2515800337 2024-11-01T17:04:21.3492104Z compile time instruction count for iteration 1 is 1170248390 2024-11-01T17:04:22.3582406Z compile time instruction count for iteration 2 is 1163927579 2024-11-01T17:04:23.4221497Z compile time instruction count for iteration 3 is 1160674259 2024-11-01T17:04:24.4201645Z compile time instruction count for iteration 4 is 1158751044 2024-11-01T17:04:25.3311004Z ++ for file in $python_programs_dir/*.py 2024-11-01T17:04:25.3314131Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/symint_sum.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv 2024-11-01T17:04:27.6399864Z collecting compile time instruction count for symint_sum 2024-11-01T17:04:41.4211734Z compile time instruction count for iteration 0 is 27983301559 2024-11-01T17:04:42.5171575Z compile time instruction count for iteration 1 is 3333858706 2024-11-01T17:04:43.6184964Z compile time instruction count for iteration 2 is 3293849451 2024-11-01T17:04:44.7171645Z compile time instruction count for iteration 3 is 3288668930 2024-11-01T17:04:45.8181897Z compile time instruction count for iteration 4 is 3286876789 2024-11-01T17:04:46.8875955Z ++ for file in $python_programs_dir/*.py 2024-11-01T17:04:46.8877675Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/update_hint_benchmark.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv 2024-11-01T17:04:49.2085986Z collecting compile time instruction count for update_hint_regression 2024-11-01T17:04:53.7313278Z compile time instruction count for iteration 0 is 24936813972 2024-11-01T17:04:54.3464335Z compile time instruction count for iteration 1 is 1763422022 2024-11-01T17:04:54.9774215Z compile time instruction count for iteration 2 is 1754445205 2024-11-01T17:04:55.5930592Z compile time instruction count for iteration 3 is 1749825482 2024-11-01T17:04:56.2058859Z compile time instruction count for iteration 4 is 1747722261 2024-11-01T17:04:57.2326532Z +++ date +%s 2024-11-01T17:04:57.2335026Z ++ end=1730480697 2024-11-01T17:04:57.2335427Z ++ runtime=578 2024-11-01T17:04:57.2336043Z ++ echo 'total time to run benchmarks is:' 2024-11-01T17:04:57.2336561Z total time to run benchmarks is: 2024-11-01T17:04:57.2337784Z ++ echo 578 2024-11-01T17:04:57.2338079Z 578 2024-11-01T17:04:57.2338642Z ++ python benchmarks/dynamo/pr_time_benchmarks/log_benchmarking_time.py 578 2024-11-01T17:04:59.0948658Z + echo 'benchmark results on current PR: ' 2024-11-01T17:04:59.0949251Z benchmark results on current PR: 2024-11-01T17:04:59.0950112Z + cat /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv 2024-11-01T17:04:59.0962136Z add_loop_eager,compile_time_instruction_count,3022834455 2024-11-01T17:04:59.0962614Z 2024-11-01T17:04:59.0963445Z add_loop_eager_dynamic,compile_time_instruction_count,5600346214 2024-11-01T17:04:59.0964004Z 2024-11-01T17:04:59.0964280Z add_loop_inductor,compile_time_instruction_count,24378418082 2024-11-01T17:04:59.0964748Z 2024-11-01T17:04:59.0965070Z add_loop_inductor_gpu,compile_time_instruction_count,23083851067 2024-11-01T17:04:59.0965543Z 2024-11-01T17:04:59.0965889Z add_loop_inductor_dynamic_gpu,compile_time_instruction_count,40475270434 2024-11-01T17:04:59.0966409Z 2024-11-01T17:04:59.0967169Z aotdispatcher_inference_nosubclass_cpu,compile_time_instruction_count,1996478859 2024-11-01T17:04:59.0967772Z 2024-11-01T17:04:59.0968168Z aotdispatcher_training_nosubclass_cpu,compile_time_instruction_count,3818120096 2024-11-01T17:04:59.0968749Z 2024-11-01T17:04:59.0969151Z aotdispatcher_inference_subclass_cpu,compile_time_instruction_count,5771607233 2024-11-01T17:04:59.0969717Z 2024-11-01T17:04:59.0970097Z aotdispatcher_training_subclass_cpu,compile_time_instruction_count,10258440935 2024-11-01T17:04:59.0970695Z 2024-11-01T17:04:59.0971032Z aotdispatcher_partitioner_cpu,compile_time_instruction_count,9008664830 2024-11-01T17:04:59.0971591Z 2024-11-01T17:04:59.0971957Z basic_modules_ListOfLinears_eager,compile_time_instruction_count,1030754764 2024-11-01T17:04:59.0972533Z 2024-11-01T17:04:59.0972911Z basic_modules_ListOfLinears_inductor,compile_time_instruction_count,19196438015 2024-11-01T17:04:59.0973488Z 2024-11-01T17:04:59.0973877Z basic_modules_ListOfLinears_inductor_gpu,compile_time_instruction_count,16849596082 2024-11-01T17:04:59.0974465Z 2024-11-01T17:04:59.0974955Z basic_modules_ListOfLinears_inductor_gpu_force_shape_pad,compile_time_instruction_count,15813889578 2024-11-01T17:04:59.0975624Z 2024-11-01T17:04:59.0975928Z sum_floordiv_regression,compile_time_instruction_count,1158751044 2024-11-01T17:04:59.0976412Z 2024-11-01T17:04:59.0976740Z symint_sum,compile_time_instruction_count,3286876789 2024-11-01T17:04:59.0977129Z 2024-11-01T17:04:59.0977421Z update_hint_regression,compile_time_instruction_count,1747722261 2024-11-01T17:04:59.0977968Z 2024-11-01T17:04:59.0978065Z ++ pwd 2024-11-01T17:04:59.0978597Z + PYTHONPATH=/var/lib/jenkins/workspace/benchmarks/dynamo/pr_time_benchmarks 2024-11-01T17:04:59.0980964Z + python benchmarks/dynamo/pr_time_benchmarks/check_results.py benchmarks/dynamo/pr_time_benchmarks/expected_results.csv /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv /var/lib/jenkins/workspace/test/test-reports/new_expected_results.csv 2024-11-01T17:05:00.5775364Z PASS: benchmark ('add_loop_eager', 'compile_time_instruction_count') pass, actual result 3022834455 -0.14% is within expected 3027000000 ±1.50% 2024-11-01T17:05:00.5777851Z 2024-11-01T17:05:00.6397338Z PASS: benchmark ('add_loop_eager_dynamic', 'compile_time_instruction_count') pass, actual result 5600346214 +0.08% is within expected 5596000000 ±2.50% 2024-11-01T17:05:00.6401427Z 2024-11-01T17:05:00.6402543Z PASS: benchmark ('add_loop_inductor', 'compile_time_instruction_count') pass, actual result 24378418082 +0.49% is within expected 24260000000 ±1.50% 2024-11-01T17:05:00.6403563Z 2024-11-01T17:05:00.6404549Z PASS: benchmark ('add_loop_inductor_dynamic_gpu', 'compile_time_instruction_count') pass, actual result 40475270434 +0.24% is within expected 40380000000 ±2.50% 2024-11-01T17:05:00.6405730Z 2024-11-01T17:05:00.6406689Z PASS: benchmark ('add_loop_inductor_gpu', 'compile_time_instruction_count') pass, actual result 23083851067 +0.32% is within expected 23010000000 ±1.50% 2024-11-01T17:05:00.6407655Z 2024-11-01T17:05:00.6408652Z PASS: benchmark ('basic_modules_ListOfLinears_eager', 'compile_time_instruction_count') pass, actual result 1030754764 +0.27% is within expected 1028000000 ±1.50% 2024-11-01T17:05:00.6409879Z 2024-11-01T17:05:00.6410912Z PASS: benchmark ('basic_modules_ListOfLinears_inductor', 'compile_time_instruction_count') pass, actual result 19196438015 +0.14% is within expected 19170000000 ±1.50% 2024-11-01T17:05:00.6411978Z 2024-11-01T17:05:00.6413131Z PASS: benchmark ('basic_modules_ListOfLinears_inductor_gpu_force_shape_pad', 'compile_time_instruction_count') pass, actual result 15813889578 +0.02% is within expected 15810000000 ±1.50% 2024-11-01T17:05:00.6414297Z 2024-11-01T17:05:00.6415348Z PASS: benchmark ('basic_modules_ListOfLinears_inductor_gpu', 'compile_time_instruction_count') pass, actual result 16849596082 +0.53% is within expected 16760000000 ±20.00% 2024-11-01T17:05:00.6416411Z 2024-11-01T17:05:00.6417468Z PASS: benchmark ('update_hint_regression', 'compile_time_instruction_count') pass, actual result 1747722261 +0.27% is within expected 1743000000 ±2.00% 2024-11-01T17:05:00.6418427Z 2024-11-01T17:05:00.6419364Z PASS: benchmark ('sum_floordiv_regression', 'compile_time_instruction_count') pass, actual result 1158751044 -0.11% is within expected 1160000000 ±1.50% 2024-11-01T17:05:00.6420333Z 2024-11-01T17:05:00.6421186Z PASS: benchmark ('symint_sum', 'compile_time_instruction_count') pass, actual result 3286876789 -0.19% is within expected 3293000000 ±1.50% 2024-11-01T17:05:00.6422069Z 2024-11-01T17:05:00.6423111Z PASS: benchmark ('aotdispatcher_inference_nosubclass_cpu', 'compile_time_instruction_count') pass, actual result 1996478859 -0.23% is within expected 2001000000 ±1.50% 2024-11-01T17:05:00.6424164Z 2024-11-01T17:05:00.6425195Z PASS: benchmark ('aotdispatcher_inference_subclass_cpu', 'compile_time_instruction_count') pass, actual result 5771607233 -0.11% is within expected 5778000000 ±1.50% 2024-11-01T17:05:00.6426239Z 2024-11-01T17:05:00.6427254Z PASS: benchmark ('aotdispatcher_partitioner_cpu', 'compile_time_instruction_count') pass, actual result 9008664830 +0.22% is within expected 8989000000 ±1.50% 2024-11-01T17:05:00.6428264Z 2024-11-01T17:05:00.6429345Z PASS: benchmark ('aotdispatcher_training_nosubclass_cpu', 'compile_time_instruction_count') pass, actual result 3818120096 -0.10% is within expected 3822000000 ±1.50% 2024-11-01T17:05:00.6430410Z 2024-11-01T17:05:00.6431459Z PASS: benchmark ('aotdispatcher_training_subclass_cpu', 'compile_time_instruction_count') pass, actual result 10258440935 -0.02% is within expected 10260000000 ±1.50% 2024-11-01T17:05:00.6432504Z 2024-11-01T17:05:00.6432706Z new expected results file content if needed: 2024-11-01T17:05:00.6433356Z add_loop_eager,compile_time_instruction_count,3022000000,0.015 2024-11-01T17:05:00.6433830Z 2024-11-01T17:05:00.6433835Z 2024-11-01T17:05:00.6433840Z 2024-11-01T17:05:00.6434157Z add_loop_eager_dynamic,compile_time_instruction_count,5600000000,0.025 2024-11-01T17:05:00.6434685Z 2024-11-01T17:05:00.6434689Z 2024-11-01T17:05:00.6434701Z 2024-11-01T17:05:00.6434997Z add_loop_inductor,compile_time_instruction_count,24370000000,0.015 2024-11-01T17:05:00.6435506Z 2024-11-01T17:05:00.6435511Z 2024-11-01T17:05:00.6435516Z 2024-11-01T17:05:00.6435884Z add_loop_inductor_dynamic_gpu,compile_time_instruction_count,40470000000,0.025 2024-11-01T17:05:00.6436548Z 2024-11-01T17:05:00.6436553Z 2024-11-01T17:05:00.6436558Z 2024-11-01T17:05:00.6436866Z add_loop_inductor_gpu,compile_time_instruction_count,23080000000,0.015 2024-11-01T17:05:00.6437376Z 2024-11-01T17:05:00.6437381Z 2024-11-01T17:05:00.6437386Z 2024-11-01T17:05:00.6437768Z basic_modules_ListOfLinears_eager,compile_time_instruction_count,1030000000,0.015 2024-11-01T17:05:00.6438354Z 2024-11-01T17:05:00.6438359Z 2024-11-01T17:05:00.6438364Z 2024-11-01T17:05:00.6438769Z basic_modules_ListOfLinears_inductor,compile_time_instruction_count,19190000000,0.015 2024-11-01T17:05:00.6439389Z 2024-11-01T17:05:00.6439394Z 2024-11-01T17:05:00.6439398Z 2024-11-01T17:05:00.6439927Z basic_modules_ListOfLinears_inductor_gpu_force_shape_pad,compile_time_instruction_count,15810000000,0.015 2024-11-01T17:05:00.6440659Z 2024-11-01T17:05:00.6440664Z 2024-11-01T17:05:00.6440669Z 2024-11-01T17:05:00.6441095Z basic_modules_ListOfLinears_inductor_gpu,compile_time_instruction_count,16840000000,0.2 2024-11-01T17:05:00.6441741Z 2024-11-01T17:05:00.6441745Z 2024-11-01T17:05:00.6441750Z 2024-11-01T17:05:00.6442072Z update_hint_regression,compile_time_instruction_count,1747000000,0.02 2024-11-01T17:05:00.6442610Z 2024-11-01T17:05:00.6442615Z 2024-11-01T17:05:00.6442620Z 2024-11-01T17:05:00.6442952Z sum_floordiv_regression,compile_time_instruction_count,1158000000,0.015 2024-11-01T17:05:00.6443501Z 2024-11-01T17:05:00.6443506Z 2024-11-01T17:05:00.6443511Z 2024-11-01T17:05:00.6443770Z symint_sum,compile_time_instruction_count,3286000000,0.015 2024-11-01T17:05:00.6444231Z 2024-11-01T17:05:00.6444236Z 2024-11-01T17:05:00.6444240Z 2024-11-01T17:05:00.6444717Z aotdispatcher_inference_nosubclass_cpu,compile_time_instruction_count,1996000000,0.015 2024-11-01T17:05:00.6445357Z 2024-11-01T17:05:00.6445362Z 2024-11-01T17:05:00.6445367Z 2024-11-01T17:05:00.6445771Z aotdispatcher_inference_subclass_cpu,compile_time_instruction_count,5771000000,0.015 2024-11-01T17:05:00.6446389Z 2024-11-01T17:05:00.6446399Z 2024-11-01T17:05:00.6446404Z 2024-11-01T17:05:00.6446768Z aotdispatcher_partitioner_cpu,compile_time_instruction_count,9008000000,0.015 2024-11-01T17:05:00.6447337Z 2024-11-01T17:05:00.6447342Z 2024-11-01T17:05:00.6447347Z 2024-11-01T17:05:00.6447751Z aotdispatcher_training_nosubclass_cpu,compile_time_instruction_count,3818000000,0.015 2024-11-01T17:05:00.6448353Z 2024-11-01T17:05:00.6448371Z 2024-11-01T17:05:00.6448376Z 2024-11-01T17:05:00.6448784Z aotdispatcher_training_subclass_cpu,compile_time_instruction_count,10250000000,0.015 2024-11-01T17:05:00.6449378Z 2024-11-01T17:05:00.6449396Z 2024-11-01T17:05:00.6449402Z 2024-11-01T17:05:00.6449406Z 2024-11-01T17:05:00.6449539Z All benchmarks passed 2024-11-01T17:05:00.9273411Z + exit 0 2024-11-01T17:05:00.9274243Z + cleanup_workspace 2024-11-01T17:05:00.9276709Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-11-01T17:05:00.9277988Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-11-01T17:05:00.9279076Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-11-01T17:05:00.9279900Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-11-01T17:05:00.9280784Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-11-01T17:05:00.9281891Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-11-01T17:05:00.9282670Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-11-01T17:05:01.3519364Z ##[group]Run cat test/**/*_toprint.log || true 2024-11-01T17:05:01.3519887Z cat test/**/*_toprint.log || true 2024-11-01T17:05:01.3529619Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T17:05:01.3530140Z env: 2024-11-01T17:05:01.3530416Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:01.3530883Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:01.3531655Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:01.3532474Z ##[endgroup] 2024-11-01T17:05:01.3605231Z cat: 'test/**/*_toprint.log': No such file or directory 2024-11-01T17:05:01.3641382Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-11-01T17:05:01.3641881Z kill "$MONITOR_SCRIPT_PID" 2024-11-01T17:05:01.3647520Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T17:05:01.3648069Z env: 2024-11-01T17:05:01.3648371Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:01.3648831Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:01.3649644Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:01.3650373Z MONITOR_SCRIPT_PID: 83424 2024-11-01T17:05:01.3650752Z ##[endgroup] 2024-11-01T17:05:01.3883696Z Prepare all required actions 2024-11-01T17:05:01.3884169Z Getting action download info 2024-11-01T17:05:01.5952813Z Download action repository 'actions/upload-artifact@v3' (SHA:ff15f0306b3f739f7b6fd43fb5d26cd321bd4de5) 2024-11-01T17:05:01.7668939Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-11-01T17:05:01.7669441Z with: 2024-11-01T17:05:01.7670025Z file-suffix: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891 2024-11-01T17:05:01.7670772Z s3-bucket: gha-artifacts 2024-11-01T17:05:01.7671143Z env: 2024-11-01T17:05:01.7671429Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:01.7671900Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:01.7672686Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:01.7673394Z ##[endgroup] 2024-11-01T17:05:01.7705473Z ##[group]Run # Remove any previous test jsons if they exist 2024-11-01T17:05:01.7706204Z # Remove any previous test jsons if they exist 2024-11-01T17:05:01.7706743Z rm -f test-jsons-*.zip 2024-11-01T17:05:01.7707342Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test/test-reports -i '*.json' 2024-11-01T17:05:01.7713141Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T17:05:01.7713742Z env: 2024-11-01T17:05:01.7714064Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:01.7714589Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:01.7715459Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:01.7716532Z FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891 2024-11-01T17:05:01.7717258Z ##[endgroup] 2024-11-01T17:05:01.7760321Z zip warning: zip file empty 2024-11-01T17:05:01.7789301Z ##[group]Run # Remove any previous test reports if they exist 2024-11-01T17:05:01.7789985Z # Remove any previous test reports if they exist 2024-11-01T17:05:01.7790564Z rm -f test-reports-*.zip 2024-11-01T17:05:01.7791246Z zip -r "test-reports-${FILE_SUFFIX}.zip" test/test-reports -i '*.xml' -i '*.csv' 2024-11-01T17:05:01.7796774Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T17:05:01.7797338Z env: 2024-11-01T17:05:01.7797638Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:01.7798145Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:01.7798976Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:01.7799999Z FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891 2024-11-01T17:05:01.7800724Z ##[endgroup] 2024-11-01T17:05:01.7844149Z adding: test/test-reports/pr_time_benchmarks_results.csv (deflated 72%) 2024-11-01T17:05:01.7845148Z adding: test/test-reports/new_expected_results.csv (deflated 78%) 2024-11-01T17:05:01.7873908Z ##[group]Run # Remove any previous usage logs if they exist 2024-11-01T17:05:01.7874565Z # Remove any previous usage logs if they exist 2024-11-01T17:05:01.7875058Z rm -f logs-*.zip 2024-11-01T17:05:01.7875742Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-11-01T17:05:01.7876709Z # so check to see if the file exists first 2024-11-01T17:05:01.7877234Z if [ -f 'usage_log.txt' ]; then 2024-11-01T17:05:01.7877782Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-11-01T17:05:01.7878291Z fi 2024-11-01T17:05:01.7878800Z if find "test/test-reports" -name "*.log" 2>/dev/null | grep -q .; then 2024-11-01T17:05:01.7879624Z  zip -r "logs-${FILE_SUFFIX}.zip" test/test-reports -i '*.log' 2024-11-01T17:05:01.7880213Z fi 2024-11-01T17:05:01.7885839Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T17:05:01.7886342Z env: 2024-11-01T17:05:01.7886625Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:01.7887089Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:01.7888043Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:01.7888994Z FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891 2024-11-01T17:05:01.7889674Z ##[endgroup] 2024-11-01T17:05:01.7932445Z adding: usage_log.txt (deflated 92%) 2024-11-01T17:05:01.8797699Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-11-01T17:05:01.8798486Z # Remove any previous debugging artifacts if they exist 2024-11-01T17:05:01.8799067Z rm -f debug-*.zip 2024-11-01T17:05:01.8799466Z if [ -d 'test/debug' ]; then 2024-11-01T17:05:01.8799993Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-11-01T17:05:01.8800490Z fi 2024-11-01T17:05:01.8805745Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T17:05:01.8806300Z env: 2024-11-01T17:05:01.8806573Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:01.8807075Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:01.8807916Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:01.8808957Z FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891 2024-11-01T17:05:01.8809801Z ##[endgroup] 2024-11-01T17:05:01.8900292Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-11-01T17:05:01.8900756Z with: 2024-11-01T17:05:01.8901021Z s3-bucket: gha-artifacts 2024-11-01T17:05:01.8901468Z s3-prefix: pytorch/pytorch/11632514903/1/artifact 2024-11-01T17:05:01.8902135Z retention-days: 14 2024-11-01T17:05:01.8902497Z if-no-files-found: warn 2024-11-01T17:05:01.8902886Z path: test-jsons-*.zip 2024-11-01T17:05:01.8903257Z name: artifact 2024-11-01T17:05:01.8903560Z region: us-east-1 2024-11-01T17:05:01.8903881Z env: 2024-11-01T17:05:01.8904175Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:01.8904664Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:01.8905501Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:01.8906213Z ##[endgroup] 2024-11-01T17:05:02.2218412Z NOTE: s3-prefix specified, ignoring name parameter 2024-11-01T17:05:02.2220186Z With the provided path, there will be 1 file uploaded 2024-11-01T17:05:02.2222073Z Uploading to s3 prefix: pytorch/pytorch/11632514903/1/artifact 2024-11-01T17:05:02.2251665Z Starting upload of test-jsons-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891.zip 2024-11-01T17:05:02.3083767Z Finished upload of test-jsons-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891.zip 2024-11-01T17:05:02.3244916Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-11-01T17:05:02.3245405Z with: 2024-11-01T17:05:02.3245682Z s3-bucket: gha-artifacts 2024-11-01T17:05:02.3246148Z s3-prefix: pytorch/pytorch/11632514903/1/artifact 2024-11-01T17:05:02.3246659Z retention-days: 14 2024-11-01T17:05:02.3247017Z if-no-files-found: error 2024-11-01T17:05:02.3247424Z path: test-reports-*.zip 2024-11-01T17:05:02.3247774Z name: artifact 2024-11-01T17:05:02.3248107Z region: us-east-1 2024-11-01T17:05:02.3248424Z env: 2024-11-01T17:05:02.3248710Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:02.3249352Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:02.3250150Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:02.3250838Z ##[endgroup] 2024-11-01T17:05:02.6529931Z NOTE: s3-prefix specified, ignoring name parameter 2024-11-01T17:05:02.6530828Z With the provided path, there will be 1 file uploaded 2024-11-01T17:05:02.6531535Z Uploading to s3 prefix: pytorch/pytorch/11632514903/1/artifact 2024-11-01T17:05:02.6562104Z Starting upload of test-reports-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891.zip 2024-11-01T17:05:02.7614442Z Finished upload of test-reports-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891.zip 2024-11-01T17:05:02.7773822Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-11-01T17:05:02.7774299Z with: 2024-11-01T17:05:02.7774576Z s3-bucket: gha-artifacts 2024-11-01T17:05:02.7775046Z s3-prefix: pytorch/pytorch/11632514903/1/artifact 2024-11-01T17:05:02.7775582Z retention-days: 14 2024-11-01T17:05:02.7775952Z if-no-files-found: ignore 2024-11-01T17:05:02.7776346Z path: logs-*.zip 2024-11-01T17:05:02.7776671Z name: artifact 2024-11-01T17:05:02.7777005Z region: us-east-1 2024-11-01T17:05:02.7777319Z env: 2024-11-01T17:05:02.7777602Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:02.7778088Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:02.7778933Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:02.7779647Z ##[endgroup] 2024-11-01T17:05:03.1061704Z NOTE: s3-prefix specified, ignoring name parameter 2024-11-01T17:05:03.1062581Z With the provided path, there will be 1 file uploaded 2024-11-01T17:05:03.1063306Z Uploading to s3 prefix: pytorch/pytorch/11632514903/1/artifact 2024-11-01T17:05:03.1094096Z Starting upload of logs-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891.zip 2024-11-01T17:05:03.2044552Z Finished upload of logs-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_32397131891.zip 2024-11-01T17:05:03.2205372Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-11-01T17:05:03.2205860Z with: 2024-11-01T17:05:03.2206131Z s3-bucket: gha-artifacts 2024-11-01T17:05:03.2206590Z s3-prefix: pytorch/pytorch/11632514903/1/artifact 2024-11-01T17:05:03.2207109Z retention-days: 14 2024-11-01T17:05:03.2207460Z if-no-files-found: ignore 2024-11-01T17:05:03.2207837Z path: debug-*.zip 2024-11-01T17:05:03.2208146Z name: artifact 2024-11-01T17:05:03.2208455Z region: us-east-1 2024-11-01T17:05:03.2208759Z env: 2024-11-01T17:05:03.2209035Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:03.2209606Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:03.2210626Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:03.2211358Z ##[endgroup] 2024-11-01T17:05:03.5436504Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-11-01T17:05:03.5598102Z ##[group]Run # shellcheck disable=SC2156 2024-11-01T17:05:03.5598602Z # shellcheck disable=SC2156 2024-11-01T17:05:03.5599441Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-11-01T17:05:03.5605455Z shell: /usr/bin/bash -e {0} 2024-11-01T17:05:03.5605825Z env: 2024-11-01T17:05:03.5606114Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:03.5606600Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:03.5607428Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:03.5608151Z ##[endgroup] 2024-11-01T17:05:03.7334097Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-11-01T17:05:03.7349974Z with: 2024-11-01T17:05:03.7350241Z env: 2024-11-01T17:05:03.7350518Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:03.7350990Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:03.7351770Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:03.7352651Z ##[endgroup] 2024-11-01T17:05:03.7373512Z ##[group]Run set -eou pipefail 2024-11-01T17:05:03.7373925Z set -eou pipefail 2024-11-01T17:05:03.7374262Z  2024-11-01T17:05:03.7374787Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-11-01T17:05:03.7375456Z for _ in $(seq 1440); do 2024-11-01T17:05:03.7375927Z  # Break if no ssh session exists anymore 2024-11-01T17:05:03.7376433Z  if [ "$(who)" = "" ]; then 2024-11-01T17:05:03.7376828Z  break 2024-11-01T17:05:03.7377182Z  fi 2024-11-01T17:05:03.7377472Z  echo "." 2024-11-01T17:05:03.7377780Z  sleep 5 2024-11-01T17:05:03.7378090Z done 2024-11-01T17:05:03.7382641Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T17:05:03.7383158Z env: 2024-11-01T17:05:03.7383427Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:03.7383897Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:03.7384667Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:03.7385352Z ##[endgroup] 2024-11-01T17:05:03.7405990Z Holding runner for 2 hours until all ssh sessions have logged out 2024-11-01T17:05:03.7452641Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-11-01T17:05:03.7453437Z # ignore expansion of "docker ps -q" since it could be empty 2024-11-01T17:05:03.7454049Z # shellcheck disable=SC2046 2024-11-01T17:05:03.7454530Z docker stop $(docker ps -q) || true 2024-11-01T17:05:03.7455015Z # Prune all of the docker images 2024-11-01T17:05:03.7455470Z docker system prune -af 2024-11-01T17:05:03.7459847Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T17:05:03.7460348Z env: 2024-11-01T17:05:03.7460620Z GIT_DEFAULT_BRANCH: main 2024-11-01T17:05:03.7461077Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-11-01T17:05:03.7461862Z DOCKER_CONTAINER_ID: b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:03.7462547Z ##[endgroup] 2024-11-01T17:05:04.2789592Z b27d0eddff79 2024-11-01T17:05:04.6118438Z Deleted Containers: 2024-11-01T17:05:04.6119037Z b27d0eddff79351d73d183c4ee702436327984856a5fc8a89704981a2a5cd4f1 2024-11-01T17:05:04.6119543Z 2024-11-01T17:05:09.5560712Z Deleted Images: 2024-11-01T17:05:09.5562736Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:bdd298b12da59246147f016e0693ffd722419941 2024-11-01T17:05:09.5565749Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:cc02ebc4db7e1029fdcc4fd31c0bc7bc340781604fac6b0edc559fbcd1ec89c4 2024-11-01T17:05:09.5567832Z deleted: sha256:235d628183ab1dfb4496bc63787f13b256e54f8c1879b1682130b9b8021e973b 2024-11-01T17:05:09.5568854Z deleted: sha256:5b6cc4f0135814f0cc1bdbb319ffc8964232ca339f5c4172f9ac50c7eaa800ab 2024-11-01T17:05:09.5569849Z deleted: sha256:283c91302d2ec145b4b644b483236ea64be8da387167beaa262b2d94ed636588 2024-11-01T17:05:09.5570841Z deleted: sha256:a26261653105d1eff44dd3eba16fdacd206c3ff79670ad5741d112b47a562c85 2024-11-01T17:05:09.5571839Z deleted: sha256:264ea8c5de742016c6619ecdb98f0f22394e3acec9274fd8d3f19a75fba0b022 2024-11-01T17:05:09.5572819Z deleted: sha256:b49a40fef57928ad5862228346e1cc67c9bc7185d424a6bcfe0568bd398394db 2024-11-01T17:05:09.5573820Z deleted: sha256:f8d4056ecb4f2dd09b3b0778fbb29c61697e7f0ae0df595b8e9ccf5e1cbabd8c 2024-11-01T17:05:09.5575150Z deleted: sha256:5fc66f028bfb2c9c319a4d0c31f19adbbb1424db3d72588fd67a2219f14be281 2024-11-01T17:05:09.5576188Z deleted: sha256:a7d57ec3b0c50fd3e26c9f5fbe7ddf5f00d44f5fb41d3cd2e4e4c3cd4bc18eb4 2024-11-01T17:05:09.5577405Z deleted: sha256:ca948114c8da2802af9006f7274e24d88973e96839070aec58b0c87deda113c6 2024-11-01T17:05:09.5578376Z deleted: sha256:1ed419f23adefbd41b6b6046884a28bd6b5dd30d7b995e987135f8cc8b9e9810 2024-11-01T17:05:09.5579455Z deleted: sha256:936f16f385621ef49424531067d6e0121e825633155be2274dea288b9662397a 2024-11-01T17:05:09.5580398Z deleted: sha256:baa5d0a9d7d6fd6b634a2be86d05ef9543f8979b409ef791fb780ea7bb18fd76 2024-11-01T17:05:09.5581388Z deleted: sha256:3dd2c8f6b70df08e7ad2e609443cdbcdff8c6a6043bfb9350bfd4d59969cb4ee 2024-11-01T17:05:09.5582388Z deleted: sha256:bc470cff2d38c5e662119bbdcfddec6899fdcfe27834a100bfa3f30da5351e9a 2024-11-01T17:05:09.5583359Z deleted: sha256:1f0cda70701a79f4d11b29190ba343c2cb557fc971946a79be4276788ef4f94b 2024-11-01T17:05:09.5584325Z deleted: sha256:562ba6fdcc9c9658487bf48b6479e2b0c437e335940e15edf131e967b08dee82 2024-11-01T17:05:09.5585288Z deleted: sha256:f3011958f7c2efac38d707a43b3ba40c38bd0fd79df167028e62c8358ee95ed8 2024-11-01T17:05:09.5586228Z deleted: sha256:c211490a9671fdc70ff745250f1387b8cd2d96747cfa02fd080e8c3d2a7141e6 2024-11-01T17:05:09.5587225Z deleted: sha256:bdbb547f6ee1075a98b60b1a3d94ca73ffacc695a225af39ea252ff4e55c8b0c 2024-11-01T17:05:09.5588205Z deleted: sha256:51bbcd3f471325e855f4b9c0765f7dc1ea74a9730d2a73627ec2289f8fbeb498 2024-11-01T17:05:09.5589370Z deleted: sha256:2bb3de3cbeee7378a4c5f16646a5aa25cf655593ef5416f515dec8fb2b074d8f 2024-11-01T17:05:09.5590375Z deleted: sha256:367d3773d653c8f5bac08f67a5ecd035b307f36b71879c041c3c5d8cc71a0a2a 2024-11-01T17:05:09.5591354Z deleted: sha256:c54c09ff282d2529482f921fbeb202667bd4b184d3458ad9577750ec1f645064 2024-11-01T17:05:09.5592307Z deleted: sha256:af6a774b2333141519b89922f5bc3d4599d0a8cbb8431050f62c149771474765 2024-11-01T17:05:09.5593280Z deleted: sha256:045b9c07c29781bc43605c78b60fdcbdcf31d103c0f5217b58f94a63d1990f45 2024-11-01T17:05:09.5594277Z deleted: sha256:13dc9ff25453cf714fb64ebf271fd0f6e963100dd12869f7aceef563d50b198f 2024-11-01T17:05:09.5595282Z deleted: sha256:3cc3211caeae4f7556de00eb224d03ed895904e1c0b9d18231d5b3e2cfb40373 2024-11-01T17:05:09.5596273Z deleted: sha256:29e078ad5e00f132916cf565501ab79b432050ebffacd5a1c012627d0457e8e5 2024-11-01T17:05:09.5597378Z deleted: sha256:ed9ec173c2b44594c41694fecf0ded4e13824fbbf34885bb3cc5e75a84ef2a0b 2024-11-01T17:05:09.5598340Z deleted: sha256:0515c12edfd7749b8729f9c22a00c8e6cb4822bf24dbb2f3efef1bb8fabbff2a 2024-11-01T17:05:09.5599323Z deleted: sha256:722baf0d6f93203e95b5da1eeda9cdbcfe450ba293e31b5b1322b3c82f7e939e 2024-11-01T17:05:09.5600313Z deleted: sha256:23f6aac99d8e4cb41b4bbf9dfc571436b53aa599167943b5b75e1b67ba087072 2024-11-01T17:05:09.5601295Z deleted: sha256:7c9d3946b1ae082f1bc92e713a5ff8b9d10d0e2623bfd3d822cce9c5d7a26d57 2024-11-01T17:05:09.5602285Z deleted: sha256:e5612b61b39e72c88cfeebbf2acecd1d8d3a86c63be4ba207da88c3c53236a98 2024-11-01T17:05:09.5603259Z deleted: sha256:8431d30aa31161360199ca92fb05dbefc3dde62ee77d0693202bf02e9268489a 2024-11-01T17:05:09.5604196Z deleted: sha256:d8141c9509f554b75d008e9510f638fa3c5f23719bbe2695768dfd343fcaad6d 2024-11-01T17:05:09.5605136Z deleted: sha256:6fa8ec2174ee8e38ed443ee59b0f1209cfae887b61c21ff8edd6873048b4a678 2024-11-01T17:05:09.5606093Z deleted: sha256:43e3675b6a676988e2064b8c4fc224ba29483ec874895cab48512b71509091c1 2024-11-01T17:05:09.5607048Z deleted: sha256:bd560c48b997b074ef9ea808078210a2c7e2111a78707466d0cbe250c5385417 2024-11-01T17:05:09.5607982Z deleted: sha256:4e916076f8764c6cd169f434901e4284e36b3547ff2b1aa33882ef58348ffeaf 2024-11-01T17:05:09.5608948Z deleted: sha256:7faed4b88fc35e4cf1a8ddd58c5a5277ccad3eccb5c22254144dd524bca0a2be 2024-11-01T17:05:09.5610218Z deleted: sha256:aea3af4dd71ac82798ed41bc88cea74432fd9c402687e910ba06304b09ee3fb8 2024-11-01T17:05:09.5611449Z deleted: sha256:f8227cc71a57e7bd654f0d6a0c866dbbdca310291c10a816a2336f269977cf11 2024-11-01T17:05:09.5612581Z deleted: sha256:90304b3e8b85729820262ab187b2e14c05bf5719c02889cf5d4fdc2d73ca777d 2024-11-01T17:05:09.5613588Z deleted: sha256:4847ecebfbfc0a8d279b072cdbc49516d6feb9dc648cc1b1b0cfded0fbb9bdfd 2024-11-01T17:05:09.5614587Z deleted: sha256:620e22498ff7f230608f83c553292bc542e75f5a518c6e5a77314d9e71a375af 2024-11-01T17:05:09.5615563Z deleted: sha256:feba4bb0cb0e319284e65867f7cfba2b76762292a384a5e7b3496ef4265ccc0f 2024-11-01T17:05:09.5616747Z deleted: sha256:e53a8ca055da69abf8f5912b4cdd0be4c5cc9dfb1b5e56a9435e98465f1a4f05 2024-11-01T17:05:09.5617706Z deleted: sha256:a30bde9c0ee571b065f45af30b399f002f2b06509010e377ee044d55a7856248 2024-11-01T17:05:09.5618789Z deleted: sha256:84cbce7f59bbc26ecc28ccae35bedba26c2403c69d260b76a8657b49b5b832d1 2024-11-01T17:05:09.5619714Z deleted: sha256:40870eba757157df00ad9b12a36972f099581f769feeb37f80cadd2b71a2d09c 2024-11-01T17:05:09.5620617Z deleted: sha256:1b6021be5f03068f8c5271e3cbedc4c9cdbb0867ea5202194a15e5353c5e853c 2024-11-01T17:05:09.5621520Z deleted: sha256:71a4fc02a427e20a4706bb07a8cf1954e46b5f85e94c74e29a298380f0a3d696 2024-11-01T17:05:09.5622417Z deleted: sha256:392d3ce86a4eee1545b22c6841dfc514e8ec4324522c538b414c8fd785fc3ca2 2024-11-01T17:05:09.5623337Z deleted: sha256:64ccbd94daef331cc326868ddf0835c7d7bedc5dc107e3b387ccff2a160594c5 2024-11-01T17:05:09.5624250Z deleted: sha256:32fef1c4380fa0fe58ab840a14da929970fc80be0e5f7b2206a489e62d435250 2024-11-01T17:05:09.5625148Z deleted: sha256:07c82b6d59598df23145ffa6fd571575d37695e40f5763d2072702f97940a48f 2024-11-01T17:05:09.5626054Z deleted: sha256:93eadd2ecf1949ad8020260b7d4dd4f780d6a00a5f8cb064afb33aa4e9cdbfa1 2024-11-01T17:05:09.5626953Z deleted: sha256:f840a34e3f099f79ee945e121dcf253677f605fdbe4325588a5905806172a389 2024-11-01T17:05:09.5627848Z deleted: sha256:65b230f616f24c8933bd918d3f59cd9a2b5a9b663e9f09948b37c7ffa0832fe3 2024-11-01T17:05:09.5628728Z deleted: sha256:352746e46159d00cc2329033c33aa7e335104c66fbd50c47cd0d97bb29614d09 2024-11-01T17:05:09.5629593Z deleted: sha256:2370d2bcfd18ad86020d4c359192f5282374ac8548682609fd772815a42edd3e 2024-11-01T17:05:09.5630498Z deleted: sha256:cbb4a3b174ac3e76ef50e8b32cdd5d48147ce428aa1d194536b3fe790ede6f39 2024-11-01T17:05:09.5631399Z deleted: sha256:4dd576898da8857338e5eb65d76181f018c8f32f21a6c29175c614d6ac3865d5 2024-11-01T17:05:09.5632276Z deleted: sha256:07ec399807b76ba4522f38c3950c8b84ede88d796fb8146338924c0612f6c2b8 2024-11-01T17:05:09.5633150Z deleted: sha256:cd53c8bd4e3e36692d13356c6242451f517cf2934ee9201448892d46ee14644d 2024-11-01T17:05:09.5634031Z deleted: sha256:84d27797cdb86969f3804e95f53821a287e431bde1ec25df4778207804e3416d 2024-11-01T17:05:09.5634911Z deleted: sha256:a9995ed28b005ed6e33b4af77dce97769076e0ac3f900eab49fd8b7dc3a01d97 2024-11-01T17:05:09.5635822Z deleted: sha256:cc7fcaf5e1579419d65a84a067d69a9dde05600c87aa686693a0419dded41988 2024-11-01T17:05:09.5636727Z deleted: sha256:151d3bef2848757567ea5e6eb43dde794c63e4a3ce75c7e06ca2e335d49f9d94 2024-11-01T17:05:09.5637630Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-11-01T17:05:09.5638531Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-11-01T17:05:09.5639417Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-11-01T17:05:09.5640307Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-11-01T17:05:09.5641195Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-11-01T17:05:09.5642100Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-11-01T17:05:09.5643015Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-11-01T17:05:09.5643937Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-11-01T17:05:09.5644861Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-11-01T17:05:09.5645779Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-11-01T17:05:09.5646717Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-11-01T17:05:09.5647271Z 2024-11-01T17:05:09.5647413Z Total reclaimed space: 39.91GB 2024-11-01T17:05:09.5730943Z Post job cleanup. 2024-11-01T17:05:09.5785951Z Post job cleanup. 2024-11-01T17:05:09.6625379Z [command]/usr/bin/git version 2024-11-01T17:05:09.6670660Z git version 2.40.1 2024-11-01T17:05:09.6710296Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/1670f6a8-3893-41f8-a2ac-6ecfb796bfe3' before making global git config changes 2024-11-01T17:05:09.6712091Z Adding repository directory to the temporary git global config as a safe directory 2024-11-01T17:05:09.6713588Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-11-01T17:05:09.6748833Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-11-01T17:05:09.6778958Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-11-01T17:05:09.7055438Z Entering 'android/libs/fbjni' 2024-11-01T17:05:09.7112628Z Entering 'third_party/FP16' 2024-11-01T17:05:09.7163535Z Entering 'third_party/FXdiv' 2024-11-01T17:05:09.7218475Z Entering 'third_party/NNPACK' 2024-11-01T17:05:09.7265935Z Entering 'third_party/NVTX' 2024-11-01T17:05:09.7320112Z Entering 'third_party/VulkanMemoryAllocator' 2024-11-01T17:05:09.7379314Z Entering 'third_party/XNNPACK' 2024-11-01T17:05:09.7445711Z Entering 'third_party/benchmark' 2024-11-01T17:05:09.7491919Z Entering 'third_party/composable_kernel' 2024-11-01T17:05:09.7555057Z Entering 'third_party/cpp-httplib' 2024-11-01T17:05:09.7609462Z Entering 'third_party/cpuinfo' 2024-11-01T17:05:09.7669046Z Entering 'third_party/cudnn_frontend' 2024-11-01T17:05:09.7720581Z Entering 'third_party/cutlass' 2024-11-01T17:05:09.7781281Z Entering 'third_party/eigen' 2024-11-01T17:05:09.7828486Z Entering 'third_party/fbgemm' 2024-11-01T17:05:09.7881841Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-11-01T17:05:09.7927294Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-11-01T17:05:09.7972356Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-11-01T17:05:09.8028380Z Entering 'third_party/fbgemm/third_party/googletest' 2024-11-01T17:05:09.8081831Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-11-01T17:05:09.8129905Z Entering 'third_party/flatbuffers' 2024-11-01T17:05:09.8185670Z Entering 'third_party/fmt' 2024-11-01T17:05:09.8241462Z Entering 'third_party/gemmlowp/gemmlowp' 2024-11-01T17:05:09.8298395Z Entering 'third_party/gloo' 2024-11-01T17:05:09.8351381Z Entering 'third_party/googletest' 2024-11-01T17:05:09.8401939Z Entering 'third_party/ideep' 2024-11-01T17:05:09.8457667Z Entering 'third_party/ideep/mkl-dnn' 2024-11-01T17:05:09.8513310Z Entering 'third_party/ittapi' 2024-11-01T17:05:09.8562920Z Entering 'third_party/kineto' 2024-11-01T17:05:09.8615724Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-11-01T17:05:09.8667383Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-11-01T17:05:09.8712136Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-11-01T17:05:09.8763597Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-11-01T17:05:09.8819359Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-11-01T17:05:09.8863328Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-11-01T17:05:09.8921795Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-11-01T17:05:09.8975775Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-11-01T17:05:09.9029137Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-11-01T17:05:09.9078945Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-11-01T17:05:09.9126756Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-11-01T17:05:09.9173065Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-11-01T17:05:09.9232212Z Entering 'third_party/mimalloc' 2024-11-01T17:05:09.9290619Z Entering 'third_party/nccl/nccl' 2024-11-01T17:05:09.9341693Z Entering 'third_party/nlohmann' 2024-11-01T17:05:09.9392851Z Entering 'third_party/onnx' 2024-11-01T17:05:09.9461058Z Entering 'third_party/onnx/third_party/pybind11' 2024-11-01T17:05:09.9518969Z Entering 'third_party/opentelemetry-cpp' 2024-11-01T17:05:09.9571335Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-11-01T17:05:09.9621548Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-11-01T17:05:09.9677112Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-11-01T17:05:09.9727501Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-11-01T17:05:09.9774192Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-11-01T17:05:09.9823796Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-11-01T17:05:09.9877793Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-11-01T17:05:09.9922200Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-11-01T17:05:09.9979451Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-11-01T17:05:10.0042314Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-11-01T17:05:10.0114404Z Entering 'third_party/pocketfft' 2024-11-01T17:05:10.0167589Z Entering 'third_party/protobuf' 2024-11-01T17:05:10.0226152Z Entering 'third_party/protobuf/third_party/benchmark' 2024-11-01T17:05:10.0279184Z Entering 'third_party/protobuf/third_party/googletest' 2024-11-01T17:05:10.0331116Z Entering 'third_party/psimd' 2024-11-01T17:05:10.0384597Z Entering 'third_party/pthreadpool' 2024-11-01T17:05:10.0440903Z Entering 'third_party/pybind11' 2024-11-01T17:05:10.0499674Z Entering 'third_party/python-peachpy' 2024-11-01T17:05:10.0549260Z Entering 'third_party/sleef' 2024-11-01T17:05:10.0601020Z Entering 'third_party/tensorpipe' 2024-11-01T17:05:10.0660310Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-11-01T17:05:10.0705826Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-11-01T17:05:10.0752297Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-11-01T17:05:10.0808821Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-11-01T17:05:10.0858294Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-11-01T17:05:10.0922713Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-11-01T17:05:10.0949782Z http.https://github.com/.extraheader 2024-11-01T17:05:10.0956861Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-11-01T17:05:10.0995093Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-11-01T17:05:10.1262517Z Entering 'android/libs/fbjni' 2024-11-01T17:05:10.1301076Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1337944Z Entering 'third_party/FP16' 2024-11-01T17:05:10.1376540Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1406612Z Entering 'third_party/FXdiv' 2024-11-01T17:05:10.1447369Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1478680Z Entering 'third_party/NNPACK' 2024-11-01T17:05:10.1517840Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1551426Z Entering 'third_party/NVTX' 2024-11-01T17:05:10.1586578Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1622287Z Entering 'third_party/VulkanMemoryAllocator' 2024-11-01T17:05:10.1657809Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1688557Z Entering 'third_party/XNNPACK' 2024-11-01T17:05:10.1722084Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1769636Z Entering 'third_party/benchmark' 2024-11-01T17:05:10.1802817Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1836138Z Entering 'third_party/composable_kernel' 2024-11-01T17:05:10.1870407Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1905364Z Entering 'third_party/cpp-httplib' 2024-11-01T17:05:10.1939743Z http.https://github.com/.extraheader 2024-11-01T17:05:10.1971824Z Entering 'third_party/cpuinfo' 2024-11-01T17:05:10.2005957Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2051509Z Entering 'third_party/cudnn_frontend' 2024-11-01T17:05:10.2088044Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2123038Z Entering 'third_party/cutlass' 2024-11-01T17:05:10.2158512Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2196834Z Entering 'third_party/eigen' 2024-11-01T17:05:10.2229707Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2267303Z Entering 'third_party/fbgemm' 2024-11-01T17:05:10.2298927Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2340997Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-11-01T17:05:10.2371071Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2408554Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-11-01T17:05:10.2444348Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2477781Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-11-01T17:05:10.2507762Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2554599Z Entering 'third_party/fbgemm/third_party/googletest' 2024-11-01T17:05:10.2589341Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2620509Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-11-01T17:05:10.2656850Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2691734Z Entering 'third_party/flatbuffers' 2024-11-01T17:05:10.2727724Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2761181Z Entering 'third_party/fmt' 2024-11-01T17:05:10.2792052Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2832380Z Entering 'third_party/gemmlowp/gemmlowp' 2024-11-01T17:05:10.2871489Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2903005Z Entering 'third_party/gloo' 2024-11-01T17:05:10.2936260Z http.https://github.com/.extraheader 2024-11-01T17:05:10.2966133Z Entering 'third_party/googletest' 2024-11-01T17:05:10.2997144Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3031723Z Entering 'third_party/ideep' 2024-11-01T17:05:10.3067770Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3098423Z Entering 'third_party/ideep/mkl-dnn' 2024-11-01T17:05:10.3129918Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3178142Z Entering 'third_party/ittapi' 2024-11-01T17:05:10.3212506Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3243206Z Entering 'third_party/kineto' 2024-11-01T17:05:10.3278263Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3309796Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-11-01T17:05:10.3343258Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3373174Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-11-01T17:05:10.3402764Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3442614Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-11-01T17:05:10.3475772Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3506959Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-11-01T17:05:10.3538878Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3578269Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-11-01T17:05:10.3610756Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3641745Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-11-01T17:05:10.3671291Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3711893Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-11-01T17:05:10.3746331Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3774478Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-11-01T17:05:10.3804111Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3844001Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-11-01T17:05:10.3880106Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3918689Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-11-01T17:05:10.3951381Z http.https://github.com/.extraheader 2024-11-01T17:05:10.3984377Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-11-01T17:05:10.4013289Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4048496Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-11-01T17:05:10.4079317Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4114330Z Entering 'third_party/mimalloc' 2024-11-01T17:05:10.4146078Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4185014Z Entering 'third_party/nccl/nccl' 2024-11-01T17:05:10.4220876Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4250486Z Entering 'third_party/nlohmann' 2024-11-01T17:05:10.4283928Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4320116Z Entering 'third_party/onnx' 2024-11-01T17:05:10.4351115Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4403752Z Entering 'third_party/onnx/third_party/pybind11' 2024-11-01T17:05:10.4441358Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4472858Z Entering 'third_party/opentelemetry-cpp' 2024-11-01T17:05:10.4504296Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4544524Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-11-01T17:05:10.4573832Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4611471Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-11-01T17:05:10.4641442Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4679338Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-11-01T17:05:10.4709044Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4749020Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-11-01T17:05:10.4779124Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4816862Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-11-01T17:05:10.4849731Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4889310Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-11-01T17:05:10.4920449Z http.https://github.com/.extraheader 2024-11-01T17:05:10.4957545Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-11-01T17:05:10.4991113Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5027944Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-11-01T17:05:10.5057703Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5093344Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-11-01T17:05:10.5129640Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5161388Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-11-01T17:05:10.5191599Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5247815Z Entering 'third_party/pocketfft' 2024-11-01T17:05:10.5282816Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5312926Z Entering 'third_party/protobuf' 2024-11-01T17:05:10.5345195Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5381357Z Entering 'third_party/protobuf/third_party/benchmark' 2024-11-01T17:05:10.5412071Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5444399Z Entering 'third_party/protobuf/third_party/googletest' 2024-11-01T17:05:10.5474232Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5510406Z Entering 'third_party/psimd' 2024-11-01T17:05:10.5542316Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5581339Z Entering 'third_party/pthreadpool' 2024-11-01T17:05:10.5621034Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5652353Z Entering 'third_party/pybind11' 2024-11-01T17:05:10.5688447Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5722463Z Entering 'third_party/python-peachpy' 2024-11-01T17:05:10.5756017Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5795723Z Entering 'third_party/sleef' 2024-11-01T17:05:10.5830740Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5862299Z Entering 'third_party/tensorpipe' 2024-11-01T17:05:10.5899877Z http.https://github.com/.extraheader 2024-11-01T17:05:10.5930518Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-11-01T17:05:10.5960183Z http.https://github.com/.extraheader 2024-11-01T17:05:10.6000081Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-11-01T17:05:10.6030074Z http.https://github.com/.extraheader 2024-11-01T17:05:10.6069823Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-11-01T17:05:10.6101612Z http.https://github.com/.extraheader 2024-11-01T17:05:10.6131852Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-11-01T17:05:10.6161354Z http.https://github.com/.extraheader 2024-11-01T17:05:10.6192831Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-11-01T17:05:10.6223280Z http.https://github.com/.extraheader 2024-11-01T17:05:10.6362609Z A job completed hook has been configured by the self-hosted runner administrator 2024-11-01T17:05:10.6382513Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-11-01T17:05:10.6387100Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-11-01T17:05:10.6387674Z ##[endgroup] 2024-11-01T17:05:16.7171352Z Cleaning up orphan processes