2024-06-26T07:02:51.8032025Z Current runner version: '2.317.0' 2024-06-26T07:02:51.8040197Z Runner name: 'gh-ci-gcp-a100-4' 2024-06-26T07:02:51.8041193Z Runner group name: 'Default' 2024-06-26T07:02:51.8042241Z Machine name: 'gh-ci-gcp-a100-4' 2024-06-26T07:02:51.8047885Z ##[group]GITHUB_TOKEN Permissions 2024-06-26T07:02:51.8050375Z Actions: read 2024-06-26T07:02:51.8051062Z Attestations: read 2024-06-26T07:02:51.8051617Z Checks: read 2024-06-26T07:02:51.8052092Z Contents: read 2024-06-26T07:02:51.8052681Z Deployments: read 2024-06-26T07:02:51.8053245Z Discussions: read 2024-06-26T07:02:51.8053899Z Issues: read 2024-06-26T07:02:51.8054436Z Metadata: read 2024-06-26T07:02:51.8054970Z Packages: read 2024-06-26T07:02:51.8055511Z Pages: read 2024-06-26T07:02:51.8056030Z PullRequests: read 2024-06-26T07:02:51.8056631Z RepositoryProjects: read 2024-06-26T07:02:51.8057291Z SecurityEvents: read 2024-06-26T07:02:51.8057899Z Statuses: read 2024-06-26T07:02:51.8058995Z ##[endgroup] 2024-06-26T07:02:51.8063317Z Secret source: Actions 2024-06-26T07:02:51.8064536Z Prepare workflow directory 2024-06-26T07:02:52.1298214Z Prepare all required actions 2024-06-26T07:02:52.1570427Z Getting action download info 2024-06-26T07:02:52.3699221Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5) 2024-06-26T07:02:53.1119026Z Download action repository 'pytorch/pytorch@main' (SHA:c422a9549d2b320b47a9391f6cbcf592b2876c88) 2024-06-26T07:02:58.0597855Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-06-26T07:02:58.4721229Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-06-26T07:02:59.0431323Z Getting action download info 2024-06-26T07:02:59.1668922Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-06-26T07:02:59.6338310Z Getting action download info 2024-06-26T07:02:59.7549583Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-06-26T07:03:00.1303914Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/129470 (b8c4c54d347aa776934c60784e35936878ef18dc) 2024-06-26T07:03:00.1307988Z ##[group] Inputs 2024-06-26T07:03:00.1308607Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-06-26T07:03:00.1309810Z test-matrix: {"include": [{"config": "inductor_torchbench_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "linux.gcp.a100"}]} 2024-06-26T07:03:00.1311919Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:03:00.1313431Z sync-tag: 2024-06-26T07:03:00.1314538Z timeout-minutes: 240 2024-06-26T07:03:00.1315006Z use-gha: anything-non-empty-to-use-gha 2024-06-26T07:03:00.1315534Z dashboard-tag: 2024-06-26T07:03:00.1315921Z s3-bucket: gha-artifacts 2024-06-26T07:03:00.1316354Z aws-role-to-assume: 2024-06-26T07:03:00.1316748Z ##[endgroup] 2024-06-26T07:03:00.1317852Z Complete job name: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-06-26T07:03:00.2063185Z A job started hook has been configured by the self-hosted runner administrator 2024-06-26T07:03:00.2248636Z ##[group]Run '/home/weiwangmeta/pre-job.sh' 2024-06-26T07:03:00.2267108Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:03:00.2267787Z ##[endgroup] 2024-06-26T07:03:00.2960239Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-06-26T07:03:00.2960962Z with: 2024-06-26T07:03:00.2961296Z submodules: recursive 2024-06-26T07:03:00.2961697Z fetch-depth: 0 2024-06-26T07:03:00.2962036Z env: 2024-06-26T07:03:00.2962336Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:03:00.2962767Z ##[endgroup] 2024-06-26T07:03:00.3195782Z ##[group]Run retry () { 2024-06-26T07:03:00.3196280Z retry () { 2024-06-26T07:03:00.3197140Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-06-26T07:03:00.3197824Z } 2024-06-26T07:03:00.3198160Z echo "${GITHUB_WORKSPACE}" 2024-06-26T07:03:00.3198673Z if [ -z "${NO_SUDO}" ]; then 2024-06-26T07:03:00.3199325Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:03:00.3199875Z else 2024-06-26T07:03:00.3200267Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:03:00.3200787Z fi 2024-06-26T07:03:00.3201205Z mkdir "${GITHUB_WORKSPACE}" 2024-06-26T07:03:00.3214307Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:03:00.3214938Z env: 2024-06-26T07:03:00.3215261Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:03:00.3215668Z NO_SUDO: 2024-06-26T07:03:00.3215970Z ##[endgroup] 2024-06-26T07:03:00.3297006Z /home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-06-26T07:03:03.1792806Z ##[group]Run malfet/checkout@silent-checkout 2024-06-26T07:03:03.1793425Z with: 2024-06-26T07:03:03.1793812Z ref: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:03:03.1794427Z fetch-depth: 0 2024-06-26T07:03:03.1794777Z submodules: recursive 2024-06-26T07:03:03.1795183Z quiet-checkout: true 2024-06-26T07:03:03.1795630Z repository: pytorch/pytorch 2024-06-26T07:03:03.1796269Z token: *** 2024-06-26T07:03:03.1796592Z ssh-strict: true 2024-06-26T07:03:03.1796981Z persist-credentials: true 2024-06-26T07:03:03.1797405Z clean: true 2024-06-26T07:03:03.1797792Z sparse-checkout-cone-mode: true 2024-06-26T07:03:03.1798245Z lfs: false 2024-06-26T07:03:03.1798596Z set-safe-directory: true 2024-06-26T07:03:03.1798995Z env: 2024-06-26T07:03:03.1799291Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:03:03.1799696Z ##[endgroup] 2024-06-26T07:03:03.3078702Z Syncing repository: pytorch/pytorch 2024-06-26T07:03:03.3080759Z ##[group]Getting Git version info 2024-06-26T07:03:03.3081767Z Working directory is '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:03:03.3082886Z [command]/usr/bin/git version 2024-06-26T07:03:03.3083327Z git version 2.25.1 2024-06-26T07:03:03.3085045Z ##[endgroup] 2024-06-26T07:03:03.3104757Z Temporarily overriding HOME='/home/weiwangmeta/actions-runner/_work/_temp/41903e7d-8e17-4578-9bca-c90c4a875828' before making global git config changes 2024-06-26T07:03:03.3106380Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:03:03.3112307Z [command]/usr/bin/git config --global --add safe.directory /home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-06-26T07:03:03.3163460Z Deleting the contents of '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:03:03.3169648Z ##[group]Initializing the repository 2024-06-26T07:03:03.3174610Z [command]/usr/bin/git init /home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-06-26T07:03:03.3221630Z Initialized empty Git repository in /home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/ 2024-06-26T07:03:03.3232958Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-06-26T07:03:03.3276491Z ##[endgroup] 2024-06-26T07:03:03.3277712Z ##[group]Disabling automatic garbage collection 2024-06-26T07:03:03.3281609Z [command]/usr/bin/git config --local gc.auto 0 2024-06-26T07:03:03.3319908Z ##[endgroup] 2024-06-26T07:03:03.3320985Z ##[group]Setting up auth 2024-06-26T07:03:03.3328819Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:03:03.3367374Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:03:03.3631633Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:03:03.3670834Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:03:03.3932497Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:03:03.3980071Z ##[endgroup] 2024-06-26T07:03:03.3981211Z ##[group]Fetching the repository 2024-06-26T07:03:03.3991441Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-06-26T07:03:05.9396249Z remote: Enumerating objects: 987612 2024-06-26T07:03:05.9397540Z remote: Enumerating objects: 990209, done. 2024-06-26T07:03:05.9404621Z remote: Counting objects: 0% (1/2597) 2024-06-26T07:03:05.9405625Z remote: Counting objects: 1% (26/2597) 2024-06-26T07:03:05.9406635Z remote: Counting objects: 2% (52/2597) 2024-06-26T07:03:05.9407380Z remote: Counting objects: 3% (78/2597) 2024-06-26T07:03:05.9408426Z remote: Counting objects: 4% (104/2597) 2024-06-26T07:03:05.9409643Z remote: Counting objects: 5% (130/2597) 2024-06-26T07:03:05.9411276Z remote: Counting objects: 6% (156/2597) 2024-06-26T07:03:05.9412433Z remote: Counting objects: 7% (182/2597) 2024-06-26T07:03:05.9413832Z remote: Counting objects: 8% (208/2597) 2024-06-26T07:03:05.9414943Z remote: Counting objects: 9% (234/2597) 2024-06-26T07:03:05.9415703Z remote: Counting objects: 10% (260/2597) 2024-06-26T07:03:05.9416750Z remote: Counting objects: 11% (286/2597) 2024-06-26T07:03:05.9417893Z remote: Counting objects: 12% (312/2597) 2024-06-26T07:03:05.9419143Z remote: Counting objects: 13% (338/2597) 2024-06-26T07:03:05.9420290Z remote: Counting objects: 14% (364/2597) 2024-06-26T07:03:05.9421001Z remote: Counting objects: 15% (390/2597) 2024-06-26T07:03:05.9422147Z remote: Counting objects: 16% (416/2597) 2024-06-26T07:03:05.9422956Z remote: Counting objects: 17% (442/2597) 2024-06-26T07:03:05.9423868Z remote: Counting objects: 18% (468/2597) 2024-06-26T07:03:05.9425225Z remote: Counting objects: 19% (494/2597) 2024-06-26T07:03:05.9425904Z remote: Counting objects: 20% (520/2597) 2024-06-26T07:03:05.9426673Z remote: Counting objects: 21% (546/2597) 2024-06-26T07:03:05.9427718Z remote: Counting objects: 22% (572/2597) 2024-06-26T07:03:05.9428924Z remote: Counting objects: 23% (598/2597) 2024-06-26T07:03:05.9429854Z remote: Counting objects: 24% (624/2597) 2024-06-26T07:03:05.9430888Z remote: Counting objects: 25% (650/2597) 2024-06-26T07:03:05.9431558Z remote: Counting objects: 26% (676/2597) 2024-06-26T07:03:05.9432537Z remote: Counting objects: 27% (702/2597) 2024-06-26T07:03:05.9433451Z remote: Counting objects: 28% (728/2597) 2024-06-26T07:03:05.9434421Z remote: Counting objects: 29% (754/2597) 2024-06-26T07:03:05.9435541Z remote: Counting objects: 30% (780/2597) 2024-06-26T07:03:05.9436361Z remote: Counting objects: 31% (806/2597) 2024-06-26T07:03:05.9436978Z remote: Counting objects: 32% (832/2597) 2024-06-26T07:03:05.9437564Z remote: Counting objects: 33% (858/2597) 2024-06-26T07:03:05.9438178Z remote: Counting objects: 34% (883/2597) 2024-06-26T07:03:05.9438838Z remote: Counting objects: 35% (909/2597) 2024-06-26T07:03:05.9439440Z remote: Counting objects: 36% (935/2597) 2024-06-26T07:03:05.9440491Z remote: Counting objects: 37% (961/2597) 2024-06-26T07:03:05.9441121Z remote: Counting objects: 38% (987/2597) 2024-06-26T07:03:05.9441749Z remote: Counting objects: 39% (1013/2597) 2024-06-26T07:03:05.9442393Z remote: Counting objects: 40% (1039/2597) 2024-06-26T07:03:05.9443014Z remote: Counting objects: 41% (1065/2597) 2024-06-26T07:03:05.9443653Z remote: Counting objects: 42% (1091/2597) 2024-06-26T07:03:05.9444430Z remote: Counting objects: 43% (1117/2597) 2024-06-26T07:03:05.9445301Z remote: Counting objects: 44% (1143/2597) 2024-06-26T07:03:05.9445916Z remote: Counting objects: 45% (1169/2597) 2024-06-26T07:03:05.9446543Z remote: Counting objects: 46% (1195/2597) 2024-06-26T07:03:05.9447172Z remote: Counting objects: 47% (1221/2597) 2024-06-26T07:03:05.9447846Z remote: Counting objects: 48% (1247/2597) 2024-06-26T07:03:05.9448483Z remote: Counting objects: 49% (1273/2597) 2024-06-26T07:03:05.9449158Z remote: Counting objects: 50% (1299/2597) 2024-06-26T07:03:05.9449792Z remote: Counting objects: 51% (1325/2597) 2024-06-26T07:03:05.9450418Z remote: Counting objects: 52% (1351/2597) 2024-06-26T07:03:05.9451049Z remote: Counting objects: 53% (1377/2597) 2024-06-26T07:03:05.9451663Z remote: Counting objects: 54% (1403/2597) 2024-06-26T07:03:05.9452296Z remote: Counting objects: 55% (1429/2597) 2024-06-26T07:03:05.9452922Z remote: Counting objects: 56% (1455/2597) 2024-06-26T07:03:05.9453556Z remote: Counting objects: 57% (1481/2597) 2024-06-26T07:03:05.9454172Z remote: Counting objects: 58% (1507/2597) 2024-06-26T07:03:05.9454800Z remote: Counting objects: 59% (1533/2597) 2024-06-26T07:03:05.9455422Z remote: Counting objects: 60% (1559/2597) 2024-06-26T07:03:05.9456036Z remote: Counting objects: 61% (1585/2597) 2024-06-26T07:03:05.9456662Z remote: Counting objects: 62% (1611/2597) 2024-06-26T07:03:05.9457290Z remote: Counting objects: 63% (1637/2597) 2024-06-26T07:03:05.9457922Z remote: Counting objects: 64% (1663/2597) 2024-06-26T07:03:05.9458535Z remote: Counting objects: 65% (1689/2597) 2024-06-26T07:03:05.9459195Z remote: Counting objects: 66% (1715/2597) 2024-06-26T07:03:05.9459823Z remote: Counting objects: 67% (1740/2597) 2024-06-26T07:03:05.9460450Z remote: Counting objects: 68% (1766/2597) 2024-06-26T07:03:05.9461071Z remote: Counting objects: 69% (1792/2597) 2024-06-26T07:03:05.9461744Z remote: Counting objects: 70% (1818/2597) 2024-06-26T07:03:05.9462402Z remote: Counting objects: 71% (1844/2597) 2024-06-26T07:03:05.9463029Z remote: Counting objects: 72% (1870/2597) 2024-06-26T07:03:05.9463640Z remote: Counting objects: 73% (1896/2597) 2024-06-26T07:03:05.9464552Z remote: Counting objects: 74% (1922/2597) 2024-06-26T07:03:05.9465182Z remote: Counting objects: 75% (1948/2597) 2024-06-26T07:03:05.9465808Z remote: Counting objects: 76% (1974/2597) 2024-06-26T07:03:05.9466422Z remote: Counting objects: 77% (2000/2597) 2024-06-26T07:03:05.9467050Z remote: Counting objects: 78% (2026/2597) 2024-06-26T07:03:05.9467671Z remote: Counting objects: 79% (2052/2597) 2024-06-26T07:03:05.9468288Z remote: Counting objects: 80% (2078/2597) 2024-06-26T07:03:05.9468938Z remote: Counting objects: 81% (2104/2597) 2024-06-26T07:03:05.9469574Z remote: Counting objects: 82% (2130/2597) 2024-06-26T07:03:05.9470196Z remote: Counting objects: 83% (2156/2597) 2024-06-26T07:03:05.9470823Z remote: Counting objects: 84% (2182/2597) 2024-06-26T07:03:05.9471432Z remote: Counting objects: 85% (2208/2597) 2024-06-26T07:03:05.9472060Z remote: Counting objects: 86% (2234/2597) 2024-06-26T07:03:05.9472682Z remote: Counting objects: 87% (2260/2597) 2024-06-26T07:03:05.9473508Z remote: Counting objects: 88% (2286/2597) 2024-06-26T07:03:05.9474130Z remote: Counting objects: 89% (2312/2597) 2024-06-26T07:03:05.9474753Z remote: Counting objects: 90% (2338/2597) 2024-06-26T07:03:05.9475377Z remote: Counting objects: 91% (2364/2597) 2024-06-26T07:03:05.9476001Z remote: Counting objects: 92% (2390/2597) 2024-06-26T07:03:05.9476611Z remote: Counting objects: 93% (2416/2597) 2024-06-26T07:03:05.9477234Z remote: Counting objects: 94% (2442/2597) 2024-06-26T07:03:05.9478000Z remote: Counting objects: 95% (2468/2597) 2024-06-26T07:03:05.9478630Z remote: Counting objects: 96% (2494/2597) 2024-06-26T07:03:05.9479299Z remote: Counting objects: 97% (2520/2597) 2024-06-26T07:03:05.9479910Z remote: Counting objects: 98% (2546/2597) 2024-06-26T07:03:05.9480536Z remote: Counting objects: 99% (2572/2597) 2024-06-26T07:03:05.9481161Z remote: Counting objects: 100% (2597/2597) 2024-06-26T07:03:05.9481841Z remote: Counting objects: 100% (2597/2597), done. 2024-06-26T07:03:05.9555041Z remote: Compressing objects: 0% (1/1074) 2024-06-26T07:03:05.9924015Z remote: Compressing objects: 1% (11/1074) 2024-06-26T07:03:06.0331236Z remote: Compressing objects: 2% (22/1074) 2024-06-26T07:03:06.0662618Z remote: Compressing objects: 3% (33/1074) 2024-06-26T07:03:06.0850350Z remote: Compressing objects: 4% (43/1074) 2024-06-26T07:03:06.1088077Z remote: Compressing objects: 5% (54/1074) 2024-06-26T07:03:06.1271034Z remote: Compressing objects: 6% (65/1074) 2024-06-26T07:03:06.1616596Z remote: Compressing objects: 7% (76/1074) 2024-06-26T07:03:06.2364762Z remote: Compressing objects: 8% (86/1074) 2024-06-26T07:03:06.3060567Z remote: Compressing objects: 9% (97/1074) 2024-06-26T07:03:06.3412700Z remote: Compressing objects: 10% (108/1074) 2024-06-26T07:03:06.3450857Z remote: Compressing objects: 11% (119/1074) 2024-06-26T07:03:06.3534861Z remote: Compressing objects: 12% (129/1074) 2024-06-26T07:03:06.3613986Z remote: Compressing objects: 13% (140/1074) 2024-06-26T07:03:06.3647110Z remote: Compressing objects: 14% (151/1074) 2024-06-26T07:03:06.3648015Z remote: Compressing objects: 15% (162/1074) 2024-06-26T07:03:06.3709716Z remote: Compressing objects: 16% (172/1074) 2024-06-26T07:03:06.3710532Z remote: Compressing objects: 17% (183/1074) 2024-06-26T07:03:06.3720609Z remote: Compressing objects: 18% (194/1074) 2024-06-26T07:03:06.3732937Z remote: Compressing objects: 19% (205/1074) 2024-06-26T07:03:06.3752173Z remote: Compressing objects: 20% (215/1074) 2024-06-26T07:03:06.4276129Z remote: Compressing objects: 21% (226/1074) 2024-06-26T07:03:06.4277556Z remote: Compressing objects: 22% (237/1074) 2024-06-26T07:03:06.4278327Z remote: Compressing objects: 23% (248/1074) 2024-06-26T07:03:06.4279129Z remote: Compressing objects: 24% (258/1074) 2024-06-26T07:03:06.4279792Z remote: Compressing objects: 25% (269/1074) 2024-06-26T07:03:06.4280528Z remote: Compressing objects: 26% (280/1074) 2024-06-26T07:03:06.4281696Z remote: Compressing objects: 27% (290/1074) 2024-06-26T07:03:06.4282743Z remote: Compressing objects: 28% (301/1074) 2024-06-26T07:03:06.4283877Z remote: Compressing objects: 29% (312/1074) 2024-06-26T07:03:06.4284983Z remote: Compressing objects: 30% (323/1074) 2024-06-26T07:03:06.4285690Z remote: Compressing objects: 31% (333/1074) 2024-06-26T07:03:06.4286326Z remote: Compressing objects: 32% (344/1074) 2024-06-26T07:03:06.4286976Z remote: Compressing objects: 33% (355/1074) 2024-06-26T07:03:06.4287627Z remote: Compressing objects: 34% (366/1074) 2024-06-26T07:03:06.4288276Z remote: Compressing objects: 35% (376/1074) 2024-06-26T07:03:06.4288978Z remote: Compressing objects: 36% (387/1074) 2024-06-26T07:03:06.4290158Z remote: Compressing objects: 37% (398/1074) 2024-06-26T07:03:06.4290830Z remote: Compressing objects: 38% (409/1074) 2024-06-26T07:03:06.4291481Z remote: Compressing objects: 39% (419/1074) 2024-06-26T07:03:06.4292116Z remote: Compressing objects: 40% (430/1074) 2024-06-26T07:03:06.4292764Z remote: Compressing objects: 41% (441/1074) 2024-06-26T07:03:06.4293415Z remote: Compressing objects: 42% (452/1074) 2024-06-26T07:03:06.4294367Z remote: Compressing objects: 43% (462/1074) 2024-06-26T07:03:06.4295021Z remote: Compressing objects: 44% (473/1074) 2024-06-26T07:03:06.4295657Z remote: Compressing objects: 45% (484/1074) 2024-06-26T07:03:06.4296343Z remote: Compressing objects: 46% (495/1074) 2024-06-26T07:03:06.4296991Z remote: Compressing objects: 47% (505/1074) 2024-06-26T07:03:06.4297639Z remote: Compressing objects: 48% (516/1074) 2024-06-26T07:03:06.4298283Z remote: Compressing objects: 49% (527/1074) 2024-06-26T07:03:06.4298994Z remote: Compressing objects: 50% (537/1074) 2024-06-26T07:03:06.4299639Z remote: Compressing objects: 51% (548/1074) 2024-06-26T07:03:06.4300274Z remote: Compressing objects: 52% (559/1074) 2024-06-26T07:03:06.4300952Z remote: Compressing objects: 53% (570/1074) 2024-06-26T07:03:06.4301767Z remote: Compressing objects: 54% (580/1074) 2024-06-26T07:03:06.4302420Z remote: Compressing objects: 55% (591/1074) 2024-06-26T07:03:06.4303066Z remote: Compressing objects: 56% (602/1074) 2024-06-26T07:03:06.4303757Z remote: Compressing objects: 57% (613/1074) 2024-06-26T07:03:06.4304906Z remote: Compressing objects: 58% (623/1074) 2024-06-26T07:03:06.4305568Z remote: Compressing objects: 59% (634/1074) 2024-06-26T07:03:06.4306206Z remote: Compressing objects: 60% (645/1074) 2024-06-26T07:03:06.4306909Z remote: Compressing objects: 61% (656/1074) 2024-06-26T07:03:06.4307647Z remote: Compressing objects: 62% (666/1074) 2024-06-26T07:03:06.4308303Z remote: Compressing objects: 63% (677/1074) 2024-06-26T07:03:06.4308984Z remote: Compressing objects: 64% (688/1074) 2024-06-26T07:03:06.4309666Z remote: Compressing objects: 65% (699/1074) 2024-06-26T07:03:06.4310321Z remote: Compressing objects: 66% (709/1074) 2024-06-26T07:03:06.4311045Z remote: Compressing objects: 67% (720/1074) 2024-06-26T07:03:06.4311722Z remote: Compressing objects: 68% (731/1074) 2024-06-26T07:03:06.4312372Z remote: Compressing objects: 69% (742/1074) 2024-06-26T07:03:06.4313014Z remote: Compressing objects: 70% (752/1074) 2024-06-26T07:03:06.4313696Z remote: Compressing objects: 71% (763/1074) 2024-06-26T07:03:06.4314338Z remote: Compressing objects: 72% (774/1074) 2024-06-26T07:03:06.4314987Z remote: Compressing objects: 73% (785/1074) 2024-06-26T07:03:06.4315675Z remote: Compressing objects: 74% (795/1074) 2024-06-26T07:03:06.4316335Z remote: Compressing objects: 75% (806/1074) 2024-06-26T07:03:06.4316969Z remote: Compressing objects: 76% (817/1074) 2024-06-26T07:03:06.4317626Z remote: Compressing objects: 77% (827/1074) 2024-06-26T07:03:06.4318267Z remote: Compressing objects: 78% (838/1074) 2024-06-26T07:03:06.4319963Z remote: Compressing objects: 79% (849/1074) 2024-06-26T07:03:06.4320656Z remote: Compressing objects: 80% (860/1074) 2024-06-26T07:03:06.4321346Z remote: Compressing objects: 81% (870/1074) 2024-06-26T07:03:06.4321999Z remote: Compressing objects: 82% (881/1074) 2024-06-26T07:03:06.4322649Z remote: Compressing objects: 83% (892/1074) 2024-06-26T07:03:06.4323319Z remote: Compressing objects: 84% (903/1074) 2024-06-26T07:03:06.4324043Z remote: Compressing objects: 85% (913/1074) 2024-06-26T07:03:06.4324692Z remote: Compressing objects: 86% (924/1074) 2024-06-26T07:03:06.4325626Z remote: Compressing objects: 87% (935/1074) 2024-06-26T07:03:06.4326273Z remote: Compressing objects: 88% (946/1074) 2024-06-26T07:03:06.4326927Z remote: Compressing objects: 89% (956/1074) 2024-06-26T07:03:06.4327579Z remote: Compressing objects: 90% (967/1074) 2024-06-26T07:03:06.4328232Z remote: Compressing objects: 91% (978/1074) 2024-06-26T07:03:06.4328872Z remote: Compressing objects: 92% (989/1074) 2024-06-26T07:03:06.4329654Z remote: Compressing objects: 93% (999/1074) 2024-06-26T07:03:06.4330314Z remote: Compressing objects: 94% (1010/1074) 2024-06-26T07:03:06.4330989Z remote: Compressing objects: 95% (1021/1074) 2024-06-26T07:03:06.4331641Z remote: Compressing objects: 96% (1032/1074) 2024-06-26T07:03:06.4332309Z remote: Compressing objects: 97% (1042/1074) 2024-06-26T07:03:06.4332994Z remote: Compressing objects: 98% (1053/1074) 2024-06-26T07:03:06.4333663Z remote: Compressing objects: 99% (1064/1074) 2024-06-26T07:03:06.4334383Z remote: Compressing objects: 100% (1074/1074) 2024-06-26T07:03:06.4335092Z remote: Compressing objects: 100% (1074/1074), done. 2024-06-26T07:03:28.7319797Z remote: Total 990209 (delta 2062), reused 1863 (delta 1522), pack-reused 987612 2024-06-26T07:04:02.6786782Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object} 2024-06-26T07:04:02.6822286Z b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:04:02.6829684Z ##[endgroup] 2024-06-26T07:04:02.6830595Z ##[group]Determining the checkout info 2024-06-26T07:04:02.6831480Z ##[endgroup] 2024-06-26T07:04:02.6832086Z ##[group]Checking out the ref 2024-06-26T07:04:02.6833898Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:04:04.2707873Z ##[endgroup] 2024-06-26T07:04:04.2708613Z ##[group]Setting up auth for fetching submodules 2024-06-26T07:04:04.2715248Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:04:04.2768563Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-06-26T07:04:04.2808534Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-06-26T07:04:04.2845346Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-06-26T07:04:04.2877611Z ##[endgroup] 2024-06-26T07:04:04.2878265Z ##[group]Fetching submodules 2024-06-26T07:04:04.2882011Z [command]/usr/bin/git submodule sync --recursive 2024-06-26T07:04:04.3193053Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-06-26T07:04:04.3490154Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-06-26T07:04:04.3495134Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-06-26T07:04:04.3503189Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-06-26T07:04:04.3509012Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-06-26T07:04:04.3514807Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-06-26T07:04:04.3519866Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-06-26T07:04:04.3525213Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-06-26T07:04:04.3531247Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-06-26T07:04:04.3537034Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-06-26T07:04:04.3542427Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-06-26T07:04:04.3548173Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-06-26T07:04:04.3554951Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-06-26T07:04:04.3559551Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-06-26T07:04:04.3564641Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-06-26T07:04:04.3570542Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-06-26T07:04:04.3575400Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-06-26T07:04:04.3581198Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:04:04.3587000Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-06-26T07:04:04.3593051Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-06-26T07:04:04.3598725Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-06-26T07:04:04.3604335Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-06-26T07:04:04.3610748Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-06-26T07:04:04.3617652Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-06-26T07:04:04.3623035Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-06-26T07:04:04.3630678Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-06-26T07:04:04.3636270Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-06-26T07:04:04.3642532Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-06-26T07:04:04.3648218Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-06-26T07:04:04.3653833Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-06-26T07:04:04.3659879Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-06-26T07:04:04.3668332Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-06-26T07:04:04.3673521Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-06-26T07:04:04.3680419Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-06-26T07:04:04.3686477Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-06-26T07:04:04.3693030Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-06-26T07:04:04.3784759Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-06-26T07:04:04.8769840Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-06-26T07:04:05.2579599Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-06-26T07:04:05.6011957Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-06-26T07:04:06.1246852Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-06-26T07:04:09.5170869Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-06-26T07:04:25.2693170Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-06-26T07:04:25.9411743Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-06-26T07:04:26.8644502Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-06-26T07:04:27.7228587Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-06-26T07:04:29.1652088Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-06-26T07:04:31.2306151Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-06-26T07:04:38.0349969Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-06-26T07:04:39.8534908Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-06-26T07:04:42.0117361Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-06-26T07:04:43.9295642Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-06-26T07:04:44.3039807Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-06-26T07:04:44.9741589Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-06-26T07:04:45.5257983Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-06-26T07:04:47.2170468Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-06-26T07:04:47.8441137Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-06-26T07:04:48.3478346Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-06-26T07:04:50.1817807Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-06-26T07:04:51.2677469Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-06-26T07:04:52.1982137Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-06-26T07:04:59.7183784Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-06-26T07:05:03.4531918Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-06-26T07:05:16.2039463Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-06-26T07:05:16.6355791Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-06-26T07:05:27.6441556Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-06-26T07:05:28.0005789Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-06-26T07:05:28.4597985Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-06-26T07:05:29.8162073Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-06-26T07:05:30.4112039Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-06-26T07:05:31.4157478Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-06-26T07:05:32.0888574Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-06-26T07:05:32.1300606Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-06-26T07:05:32.1766121Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-06-26T07:05:32.2317257Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-06-26T07:05:32.3042357Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-06-26T07:05:33.4145285Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-06-26T07:05:33.4684909Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-06-26T07:05:33.5430708Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-06-26T07:05:33.6870174Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-06-26T07:05:33.7535963Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a' 2024-06-26T07:05:34.3491899Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-06-26T07:05:34.6678324Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-06-26T07:05:34.7824165Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-06-26T07:05:34.7884840Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:05:34.7889222Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:05:34.7896920Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:05:34.7903804Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:05:34.7909131Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:05:34.7961759Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-06-26T07:05:36.1549508Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-06-26T07:05:37.0085605Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-06-26T07:05:39.0631220Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-06-26T07:05:40.6728069Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-06-26T07:05:41.2852241Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-06-26T07:05:41.4242732Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-06-26T07:05:41.9004764Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-06-26T07:05:41.9970667Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-06-26T07:05:42.0347345Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-06-26T07:05:42.1912424Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-06-26T07:05:42.2592649Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-06-26T07:05:42.2979365Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-06-26T07:05:42.3719246Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-06-26T07:05:42.4257878Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-06-26T07:05:42.5050685Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:05:42.5479125Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-06-26T07:05:42.5529479Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-06-26T07:05:42.5575209Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-06-26T07:05:58.0506233Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-06-26T07:05:58.1009380Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-06-26T07:05:58.2339981Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a' 2024-06-26T07:05:58.2398748Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:05:58.2403629Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:05:58.2409712Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:05:58.2461510Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-06-26T07:05:59.2141621Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-06-26T07:06:01.1628696Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-06-26T07:06:02.9098312Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-06-26T07:06:02.9158731Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:06:02.9165341Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:06:02.9171468Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:06:02.9177426Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:06:02.9182384Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:06:02.9188364Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:06:02.9194390Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:06:02.9200377Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:06:02.9253365Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-06-26T07:06:04.3433134Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-06-26T07:06:05.0680609Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-06-26T07:06:06.9880946Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-06-26T07:06:07.5850759Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-06-26T07:06:08.4195684Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-06-26T07:06:10.0822497Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-06-26T07:06:17.6157395Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-06-26T07:06:18.2864199Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-06-26T07:06:18.3302376Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-06-26T07:06:18.3969628Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-06-26T07:06:18.4350735Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-06-26T07:06:18.4404243Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:06:18.4454488Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-06-26T07:06:19.0540192Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-06-26T07:06:19.0975997Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-06-26T07:06:19.1714745Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-06-26T07:06:19.3176390Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-06-26T07:06:19.3588164Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-06-26T07:06:19.4237976Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-06-26T07:06:19.5136417Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-06-26T07:06:19.5826775Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-06-26T07:06:19.6376334Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-06-26T07:06:19.7848167Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-06-26T07:06:20.1842543Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781' 2024-06-26T07:06:20.1925517Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-06-26T07:06:20.1928184Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-06-26T07:06:20.1999334Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-06-26T07:06:20.8590087Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-06-26T07:06:22.2553699Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-06-26T07:06:22.3193043Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-06-26T07:06:22.4227388Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-06-26T07:06:22.4285851Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:06:22.4291472Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:06:22.4296883Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:06:22.4302952Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:06:22.4309217Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:06:22.4315252Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:06:22.4321067Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:06:22.4326031Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:06:22.4379090Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-06-26T07:06:23.0877158Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-06-26T07:06:24.6758643Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-06-26T07:06:25.2606317Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-06-26T07:06:32.7636552Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-06-26T07:06:33.2852977Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-06-26T07:06:33.7188456Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-06-26T07:06:34.2569694Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-06-26T07:06:44.7530957Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-06-26T07:06:44.8235242Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-06-26T07:06:44.8644592Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-06-26T07:06:45.0044553Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-06-26T07:06:45.0425553Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-06-26T07:06:45.0818780Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-06-26T07:06:45.1215569Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-06-26T07:06:45.1269967Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:06:45.1274785Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:06:45.1322670Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-06-26T07:06:48.3415597Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-06-26T07:06:50.2756028Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-06-26T07:06:50.3524270Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:06:50.8477227Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-06-26T07:06:50.8913105Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-06-26T07:06:51.2259628Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-06-26T07:06:51.2323697Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:06:51.2329219Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-06-26T07:06:51.2384491Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-06-26T07:06:51.9271170Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-06-26T07:06:53.5733538Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-06-26T07:06:53.6798147Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-06-26T07:06:53.7171544Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-06-26T07:06:53.7579345Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-06-26T07:06:53.8235707Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-06-26T07:06:53.8851918Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-06-26T07:06:53.9597681Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-06-26T07:06:54.0154391Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-06-26T07:06:54.0208048Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:06:54.0213143Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:06:54.0219379Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:06:54.0224991Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:06:54.0274298Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-06-26T07:06:55.6283094Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-06-26T07:06:56.1339611Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-06-26T07:06:57.5935028Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-06-26T07:06:58.9915953Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-06-26T07:06:59.0307013Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-06-26T07:06:59.1348981Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-06-26T07:06:59.1902153Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-06-26T07:06:59.1966291Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:06:59.2013755Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-06-26T07:06:59.6657154Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-06-26T07:06:59.6756822Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-06-26T07:06:59.7057151Z Entering 'android/libs/fbjni' 2024-06-26T07:06:59.7101394Z Entering 'third_party/FP16' 2024-06-26T07:06:59.7143798Z Entering 'third_party/FXdiv' 2024-06-26T07:06:59.7186937Z Entering 'third_party/NNPACK' 2024-06-26T07:06:59.7229935Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:06:59.7272430Z Entering 'third_party/XNNPACK' 2024-06-26T07:06:59.7335146Z Entering 'third_party/benchmark' 2024-06-26T07:06:59.7378150Z Entering 'third_party/cpp-httplib' 2024-06-26T07:06:59.7420395Z Entering 'third_party/cpuinfo' 2024-06-26T07:06:59.7463148Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:06:59.7507053Z Entering 'third_party/cutlass' 2024-06-26T07:06:59.7559853Z Entering 'third_party/eigen' 2024-06-26T07:06:59.7606824Z Entering 'third_party/fbgemm' 2024-06-26T07:06:59.7650476Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:06:59.7692934Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:06:59.7735810Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:06:59.7787002Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:06:59.7829495Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:06:59.7874248Z Entering 'third_party/flatbuffers' 2024-06-26T07:06:59.7922133Z Entering 'third_party/fmt' 2024-06-26T07:06:59.7965923Z Entering 'third_party/foxi' 2024-06-26T07:06:59.8009813Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:06:59.8052783Z Entering 'third_party/gloo' 2024-06-26T07:06:59.8095073Z Entering 'third_party/googletest' 2024-06-26T07:06:59.8139646Z Entering 'third_party/ideep' 2024-06-26T07:06:59.8181845Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:06:59.8236896Z Entering 'third_party/ittapi' 2024-06-26T07:06:59.8280382Z Entering 'third_party/kineto' 2024-06-26T07:06:59.8321847Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:06:59.8363941Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:06:59.8408316Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:06:59.8451401Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:06:59.8493491Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:06:59.8535656Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:06:59.8583132Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:06:59.8626340Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:06:59.8669009Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:06:59.8714797Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:06:59.8759806Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:06:59.8805295Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:06:59.8851139Z Entering 'third_party/mimalloc' 2024-06-26T07:06:59.8895873Z Entering 'third_party/nccl/nccl' 2024-06-26T07:06:59.8942416Z Entering 'third_party/nlohmann' 2024-06-26T07:06:59.8987914Z Entering 'third_party/onnx' 2024-06-26T07:06:59.9052190Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:06:59.9096091Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:06:59.9141415Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:06:59.9189655Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:06:59.9232106Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:06:59.9275449Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:06:59.9318794Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:06:59.9364531Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:06:59.9406646Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:06:59.9448061Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:06:59.9489073Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:06:59.9533505Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:06:59.9581938Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:06:59.9652646Z Entering 'third_party/pocketfft' 2024-06-26T07:06:59.9699478Z Entering 'third_party/protobuf' 2024-06-26T07:06:59.9747662Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:06:59.9790564Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:06:59.9836962Z Entering 'third_party/psimd' 2024-06-26T07:06:59.9880949Z Entering 'third_party/pthreadpool' 2024-06-26T07:06:59.9927000Z Entering 'third_party/pybind11' 2024-06-26T07:06:59.9971224Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:00.0015076Z Entering 'third_party/sleef' 2024-06-26T07:07:00.0063023Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:00.0108525Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:00.0151388Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:00.0193699Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:00.0234713Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:00.0275309Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:00.0341810Z ##[endgroup] 2024-06-26T07:07:00.0346524Z ##[group]Persisting credentials for submodules 2024-06-26T07:07:00.0350673Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-06-26T07:07:00.0654291Z Entering 'android/libs/fbjni' 2024-06-26T07:07:00.0704619Z Entering 'third_party/FP16' 2024-06-26T07:07:00.0753589Z Entering 'third_party/FXdiv' 2024-06-26T07:07:00.0805725Z Entering 'third_party/NNPACK' 2024-06-26T07:07:00.0858703Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:00.0909145Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:00.0980717Z Entering 'third_party/benchmark' 2024-06-26T07:07:00.1031250Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:00.1079205Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:00.1128519Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:00.1178044Z Entering 'third_party/cutlass' 2024-06-26T07:07:00.1237224Z Entering 'third_party/eigen' 2024-06-26T07:07:00.1287214Z Entering 'third_party/fbgemm' 2024-06-26T07:07:00.1335651Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:00.1381701Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:00.1429979Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:00.1485821Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:00.1533479Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:00.1582317Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:00.1633978Z Entering 'third_party/fmt' 2024-06-26T07:07:00.1680912Z Entering 'third_party/foxi' 2024-06-26T07:07:00.1729065Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:00.1777495Z Entering 'third_party/gloo' 2024-06-26T07:07:00.1826090Z Entering 'third_party/googletest' 2024-06-26T07:07:00.1873901Z Entering 'third_party/ideep' 2024-06-26T07:07:00.1919950Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:00.1975654Z Entering 'third_party/ittapi' 2024-06-26T07:07:00.2023403Z Entering 'third_party/kineto' 2024-06-26T07:07:00.2071126Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:00.2120626Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:00.2170092Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:00.2221160Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:00.2267351Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:00.2313457Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:00.2362501Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:00.2410226Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:00.2456838Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:00.2506950Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:00.2557283Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:00.2604948Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:00.2654882Z Entering 'third_party/mimalloc' 2024-06-26T07:07:00.2704147Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:00.2754332Z Entering 'third_party/nlohmann' 2024-06-26T07:07:00.2802982Z Entering 'third_party/onnx' 2024-06-26T07:07:00.2871549Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:00.2920400Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:00.2972091Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:00.3022721Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:00.3070428Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:00.3117423Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:00.3165375Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:00.3214207Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:00.3260827Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:00.3307694Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:00.3353987Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:00.3404163Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:00.3453277Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:00.3526158Z Entering 'third_party/pocketfft' 2024-06-26T07:07:00.3575677Z Entering 'third_party/protobuf' 2024-06-26T07:07:00.3628200Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:00.3674801Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:00.3723321Z Entering 'third_party/psimd' 2024-06-26T07:07:00.3770169Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:00.3818123Z Entering 'third_party/pybind11' 2024-06-26T07:07:00.3867922Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:00.3917018Z Entering 'third_party/sleef' 2024-06-26T07:07:00.3964542Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:00.4012936Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:00.4060950Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:00.4108396Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:00.4156393Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:00.4201029Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:00.4272379Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-06-26T07:07:00.4565155Z Entering 'android/libs/fbjni' 2024-06-26T07:07:00.4610611Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-06-26T07:07:00.4630965Z Entering 'third_party/FP16' 2024-06-26T07:07:00.4675092Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-06-26T07:07:00.4695237Z Entering 'third_party/FXdiv' 2024-06-26T07:07:00.4738587Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-06-26T07:07:00.4760286Z Entering 'third_party/NNPACK' 2024-06-26T07:07:00.4803945Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-06-26T07:07:00.4824456Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:00.4868347Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-06-26T07:07:00.4888501Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:00.4932031Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-06-26T07:07:00.4972549Z Entering 'third_party/benchmark' 2024-06-26T07:07:00.5016490Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:00.5038142Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:00.5083825Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-06-26T07:07:00.5104337Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:00.5149125Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:07:00.5170001Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:00.5214894Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-06-26T07:07:00.5237144Z Entering 'third_party/cutlass' 2024-06-26T07:07:00.5282649Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:07:00.5312468Z Entering 'third_party/eigen' 2024-06-26T07:07:00.5360443Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-06-26T07:07:00.5383568Z Entering 'third_party/fbgemm' 2024-06-26T07:07:00.5429039Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-06-26T07:07:00.5448678Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:00.5494786Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-06-26T07:07:00.5515229Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:00.5558929Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:07:00.5579931Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:00.5623914Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:07:00.5651902Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:00.5695746Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:00.5715346Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:00.5759364Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-06-26T07:07:00.5781174Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:00.5827988Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-06-26T07:07:00.5851788Z Entering 'third_party/fmt' 2024-06-26T07:07:00.5895260Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:07:00.5917199Z Entering 'third_party/foxi' 2024-06-26T07:07:00.5958294Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-06-26T07:07:00.5979750Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:00.6023304Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-06-26T07:07:00.6044027Z Entering 'third_party/gloo' 2024-06-26T07:07:00.6090302Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-06-26T07:07:00.6113774Z Entering 'third_party/googletest' 2024-06-26T07:07:00.6157742Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:00.6178665Z Entering 'third_party/ideep' 2024-06-26T07:07:00.6223204Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-06-26T07:07:00.6242768Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:00.6286570Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-06-26T07:07:00.6316265Z Entering 'third_party/ittapi' 2024-06-26T07:07:00.6358895Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-06-26T07:07:00.6382881Z Entering 'third_party/kineto' 2024-06-26T07:07:00.6426137Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-06-26T07:07:00.6447143Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:00.6493375Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-06-26T07:07:00.6512343Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:00.6558043Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-06-26T07:07:00.6578198Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:00.6622832Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-06-26T07:07:00.6641191Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:00.6685146Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:07:00.6703919Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:00.6749098Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-06-26T07:07:00.6767152Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:00.6810761Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-06-26T07:07:00.6832822Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:00.6876277Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-06-26T07:07:00.6895502Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:00.6939529Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:00.6960495Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:00.7004008Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-06-26T07:07:00.7025030Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:00.7068136Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-06-26T07:07:00.7090248Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:00.7133669Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-06-26T07:07:00.7153816Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:00.7197310Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-06-26T07:07:00.7219666Z Entering 'third_party/mimalloc' 2024-06-26T07:07:00.7266060Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-06-26T07:07:00.7287426Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:00.7331511Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-06-26T07:07:00.7352220Z Entering 'third_party/nlohmann' 2024-06-26T07:07:00.7397112Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-06-26T07:07:00.7417954Z Entering 'third_party/onnx' 2024-06-26T07:07:00.7462264Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-06-26T07:07:00.7501189Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:00.7546515Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:00.7567075Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:00.7611365Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:00.7635349Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:00.7680750Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-06-26T07:07:00.7702566Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:00.7746241Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:00.7765724Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:00.7808852Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:00.7828408Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:00.7871010Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-06-26T07:07:00.7890434Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:00.7936077Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-06-26T07:07:00.7956433Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:00.7999498Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-06-26T07:07:00.8018630Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:00.8061933Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-06-26T07:07:00.8081422Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:00.8126136Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-06-26T07:07:00.8145769Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:00.8191402Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-06-26T07:07:00.8213476Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:00.8256876Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-06-26T07:07:00.8278446Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:00.8322813Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-06-26T07:07:00.8367570Z Entering 'third_party/pocketfft' 2024-06-26T07:07:00.8414958Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-06-26T07:07:00.8435223Z Entering 'third_party/protobuf' 2024-06-26T07:07:00.8478649Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-06-26T07:07:00.8504526Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:00.8547476Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:00.8566409Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:00.8610406Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:00.8632729Z Entering 'third_party/psimd' 2024-06-26T07:07:00.8678980Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-06-26T07:07:00.8698844Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:00.8743304Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-06-26T07:07:00.8762553Z Entering 'third_party/pybind11' 2024-06-26T07:07:00.8807859Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:00.8829414Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:00.8874176Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-06-26T07:07:00.8893662Z Entering 'third_party/sleef' 2024-06-26T07:07:00.8938684Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-06-26T07:07:00.8960708Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:00.9006108Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-06-26T07:07:00.9025752Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:00.9069265Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:00.9087588Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:00.9131802Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-06-26T07:07:00.9151431Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:00.9194957Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-06-26T07:07:00.9214639Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:00.9258209Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:00.9276759Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:00.9324340Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-06-26T07:07:00.9722267Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-06-26T07:07:01.0025461Z Entering 'android/libs/fbjni' 2024-06-26T07:07:01.0069684Z Entering 'third_party/FP16' 2024-06-26T07:07:01.0112579Z Entering 'third_party/FXdiv' 2024-06-26T07:07:01.0156015Z Entering 'third_party/NNPACK' 2024-06-26T07:07:01.0199426Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:01.0241862Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:01.0307989Z Entering 'third_party/benchmark' 2024-06-26T07:07:01.0353407Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:01.0395749Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:01.0438980Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:01.0481258Z Entering 'third_party/cutlass' 2024-06-26T07:07:01.0535939Z Entering 'third_party/eigen' 2024-06-26T07:07:01.0582910Z Entering 'third_party/fbgemm' 2024-06-26T07:07:01.0625397Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:01.0666545Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:01.0707915Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:01.0757515Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:01.0800168Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:01.0842337Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:01.0887899Z Entering 'third_party/fmt' 2024-06-26T07:07:01.0930806Z Entering 'third_party/foxi' 2024-06-26T07:07:01.0974296Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:01.1017707Z Entering 'third_party/gloo' 2024-06-26T07:07:01.1060912Z Entering 'third_party/googletest' 2024-06-26T07:07:01.1103399Z Entering 'third_party/ideep' 2024-06-26T07:07:01.1144615Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:01.1196797Z Entering 'third_party/ittapi' 2024-06-26T07:07:01.1239329Z Entering 'third_party/kineto' 2024-06-26T07:07:01.1281254Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:01.1321717Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:01.1364004Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:01.1404433Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:01.1447881Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:01.1488222Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:01.1531722Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:01.1572612Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:01.1614033Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:01.1657108Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:01.1700480Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:01.1742548Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:01.1786332Z Entering 'third_party/mimalloc' 2024-06-26T07:07:01.1829969Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:01.1873076Z Entering 'third_party/nlohmann' 2024-06-26T07:07:01.1917871Z Entering 'third_party/onnx' 2024-06-26T07:07:01.1980364Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:01.2023768Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:01.2070393Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:01.2116142Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:01.2157816Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:01.2199890Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:01.2240245Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:01.2283899Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:01.2323474Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:01.2365450Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:01.2406084Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:01.2468174Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:01.2509710Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:01.2576648Z Entering 'third_party/pocketfft' 2024-06-26T07:07:01.2620748Z Entering 'third_party/protobuf' 2024-06-26T07:07:01.2665888Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:01.2706808Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:01.2751050Z Entering 'third_party/psimd' 2024-06-26T07:07:01.2795260Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:01.2838791Z Entering 'third_party/pybind11' 2024-06-26T07:07:01.2881978Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:01.2924852Z Entering 'third_party/sleef' 2024-06-26T07:07:01.2967931Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:01.3013659Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:01.3054780Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:01.3096858Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:01.3139919Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:01.3179559Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:01.3241987Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-06-26T07:07:01.3531802Z Entering 'android/libs/fbjni' 2024-06-26T07:07:01.3575123Z Entering 'third_party/FP16' 2024-06-26T07:07:01.3618091Z Entering 'third_party/FXdiv' 2024-06-26T07:07:01.3660418Z Entering 'third_party/NNPACK' 2024-06-26T07:07:01.3703762Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:01.3747350Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:01.3810417Z Entering 'third_party/benchmark' 2024-06-26T07:07:01.3852953Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:01.3896300Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:01.3940559Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:01.3983933Z Entering 'third_party/cutlass' 2024-06-26T07:07:01.4038513Z Entering 'third_party/eigen' 2024-06-26T07:07:01.4084248Z Entering 'third_party/fbgemm' 2024-06-26T07:07:01.4127956Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:01.4169786Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:01.4210634Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:01.4259576Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:01.4300785Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:01.4345125Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:01.4393027Z Entering 'third_party/fmt' 2024-06-26T07:07:01.4435753Z Entering 'third_party/foxi' 2024-06-26T07:07:01.4478241Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:01.4520425Z Entering 'third_party/gloo' 2024-06-26T07:07:01.4561485Z Entering 'third_party/googletest' 2024-06-26T07:07:01.4603114Z Entering 'third_party/ideep' 2024-06-26T07:07:01.4642771Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:01.4693540Z Entering 'third_party/ittapi' 2024-06-26T07:07:01.4734989Z Entering 'third_party/kineto' 2024-06-26T07:07:01.4779050Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:01.4819719Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:01.4861877Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:01.4904622Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:01.4947036Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:01.4987035Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:01.5030700Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:01.5072725Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:01.5114141Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:01.5156152Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:01.5199094Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:01.5240354Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:01.5285141Z Entering 'third_party/mimalloc' 2024-06-26T07:07:01.5328882Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:01.5374694Z Entering 'third_party/nlohmann' 2024-06-26T07:07:01.5417079Z Entering 'third_party/onnx' 2024-06-26T07:07:01.5478758Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:01.5522121Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:01.5568306Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:01.5611961Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:01.5654149Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:01.5696861Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:01.5737954Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:01.5780271Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:01.5821083Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:01.5860993Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:01.5900671Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:01.5944156Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:01.5988902Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:01.6056213Z Entering 'third_party/pocketfft' 2024-06-26T07:07:01.6100422Z Entering 'third_party/protobuf' 2024-06-26T07:07:01.6146700Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:01.6187988Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:01.6233882Z Entering 'third_party/psimd' 2024-06-26T07:07:01.6278012Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:01.6320908Z Entering 'third_party/pybind11' 2024-06-26T07:07:01.6363709Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:01.6408204Z Entering 'third_party/sleef' 2024-06-26T07:07:01.6450792Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:01.6493575Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:01.6535196Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:01.6576511Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:01.6618689Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:01.6657788Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:01.6718216Z ##[endgroup] 2024-06-26T07:07:01.6776235Z [command]/usr/bin/git log -1 --format='%H' 2024-06-26T07:07:01.6810549Z 'b8c4c54d347aa776934c60784e35936878ef18dc' 2024-06-26T07:07:01.7120101Z Prepare all required actions 2024-06-26T07:07:01.7121214Z Getting action download info 2024-06-26T07:07:02.0180218Z ##[group]Run ./.github/actions/setup-linux 2024-06-26T07:07:02.0180776Z env: 2024-06-26T07:07:02.0181095Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:02.0181499Z ##[endgroup] 2024-06-26T07:07:02.0256936Z ##[group]Run set -euo pipefail 2024-06-26T07:07:02.0257488Z set -euo pipefail 2024-06-26T07:07:02.0257934Z function get_ec2_metadata() { 2024-06-26T07:07:02.0258589Z  # Pulled from instance metadata endpoint for EC2 2024-06-26T07:07:02.0259732Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-06-26T07:07:02.0260697Z  category=$1 2024-06-26T07:07:02.0261314Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-06-26T07:07:02.0262087Z  runner_name_str=gh-ci-gcp-a100-4 2024-06-26T07:07:02.0262719Z  if [[ -f /.inarc ]]; then 2024-06-26T07:07:02.0263303Z  echo "ARC Runner, no info on ec2 metadata" 2024-06-26T07:07:02.0264235Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-06-26T07:07:02.0265105Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-06-26T07:07:02.0265859Z  else 2024-06-26T07:07:02.0266453Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-06-26T07:07:02.0267146Z  fi 2024-06-26T07:07:02.0267471Z } 2024-06-26T07:07:02.0267889Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-06-26T07:07:02.0268583Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-06-26T07:07:02.0269401Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-06-26T07:07:02.0270092Z echo "system info $(uname -a)" 2024-06-26T07:07:02.0284505Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:02.0285122Z env: 2024-06-26T07:07:02.0285444Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:02.0285855Z ##[endgroup] 2024-06-26T07:07:02.0341067Z ami-id: Runner is from Google Cloud Platform, No info on ec2 metadata 2024-06-26T07:07:02.0347696Z instance-id: Runner is from Google Cloud Platform, No info on ec2 metadata 2024-06-26T07:07:02.0354066Z instance-type: Runner is from Google Cloud Platform, No info on ec2 metadata 2024-06-26T07:07:02.0366050Z system info Linux gh-ci-gcp-a100-4 5.15.0-1047-gcp #55~20.04.1-Ubuntu SMP Wed Nov 15 11:38:25 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux 2024-06-26T07:07:02.0392680Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:07:02.0393846Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:07:02.0407093Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:02.0407704Z env: 2024-06-26T07:07:02.0408005Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:02.0408409Z ##[endgroup] 2024-06-26T07:07:02.0505595Z ##[group]Run if systemctl is-active --quiet docker; then 2024-06-26T07:07:02.0506332Z if systemctl is-active --quiet docker; then 2024-06-26T07:07:02.0507001Z  echo "Docker daemon is running..."; 2024-06-26T07:07:02.0507540Z else 2024-06-26T07:07:02.0508114Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-06-26T07:07:02.0508824Z fi 2024-06-26T07:07:02.0521345Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:02.0521950Z env: 2024-06-26T07:07:02.0522252Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:02.0522658Z ##[endgroup] 2024-06-26T07:07:02.0602886Z Docker daemon is running... 2024-06-26T07:07:02.0662383Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:07:02.0663223Z with: 2024-06-26T07:07:02.0663633Z shell: bash 2024-06-26T07:07:02.0664418Z timeout_minutes: 5 2024-06-26T07:07:02.0745587Z max_attempts: 3 2024-06-26T07:07:02.0746398Z retry_wait_seconds: 30 2024-06-26T07:07:02.0748218Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-06-26T07:07:02.0750142Z polling_interval_seconds: 1 2024-06-26T07:07:02.0750561Z warning_on_retry: true 2024-06-26T07:07:02.0750987Z continue_on_error: false 2024-06-26T07:07:02.0751357Z env: 2024-06-26T07:07:02.0751648Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:02.0752062Z AWS_RETRY_MODE: standard 2024-06-26T07:07:02.0752460Z AWS_MAX_ATTEMPTS: 5 2024-06-26T07:07:02.0752837Z AWS_DEFAULT_REGION: us-east-1 2024-06-26T07:07:02.0753266Z ##[endgroup] 2024-06-26T07:07:04.2935755Z WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json. 2024-06-26T07:07:04.2937638Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:04.2939640Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:04.2940861Z 2024-06-26T07:07:04.2941103Z Login Succeeded 2024-06-26T07:07:05.1449804Z Command completed after 1 attempt(s). 2024-06-26T07:07:05.1520352Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:05.1521261Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:05.1522099Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:05.1535718Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:05.1536334Z env: 2024-06-26T07:07:05.1536643Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:05.1537055Z ##[endgroup] 2024-06-26T07:07:05.1638541Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:07:05.1639514Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:07:05.1640236Z # shellcheck disable=SC2046 2024-06-26T07:07:05.1640805Z docker stop $(docker ps -q) || true 2024-06-26T07:07:05.1641365Z # Prune all of the docker images 2024-06-26T07:07:05.1641901Z docker system prune -af 2024-06-26T07:07:05.1654486Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:05.1655094Z env: 2024-06-26T07:07:05.1655416Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:05.1655819Z ##[endgroup] 2024-06-26T07:07:05.2091946Z "docker stop" requires at least 1 argument. 2024-06-26T07:07:05.2093186Z See 'docker stop --help'. 2024-06-26T07:07:05.2093480Z 2024-06-26T07:07:05.2093744Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-06-26T07:07:05.2094215Z 2024-06-26T07:07:05.2094382Z Stop one or more running containers 2024-06-26T07:07:05.2329753Z Total reclaimed space: 0B 2024-06-26T07:07:05.2377692Z ##[group]Run set +e 2024-06-26T07:07:05.2378112Z set +e 2024-06-26T07:07:05.2378453Z set -x 2024-06-26T07:07:05.2378787Z  2024-06-26T07:07:05.2379179Z PT_DOMAIN=download.pytorch.org 2024-06-26T07:07:05.2380392Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-06-26T07:07:05.2381749Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-06-26T07:07:05.2382699Z # one is returned at random 2024-06-26T07:07:05.2383346Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-06-26T07:07:05.2384709Z  2024-06-26T07:07:05.2385267Z if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:07:05.2386049Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-06-26T07:07:05.2386997Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-06-26T07:07:05.2387659Z  2024-06-26T07:07:05.2388026Z  if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:07:05.2388688Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-06-26T07:07:05.2389689Z  exit 1 2024-06-26T07:07:05.2390043Z  fi 2024-06-26T07:07:05.2390361Z fi 2024-06-26T07:07:05.2390670Z  2024-06-26T07:07:05.2391079Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-06-26T07:07:05.2391737Z  # Clean up any old records first 2024-06-26T07:07:05.2392343Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-06-26T07:07:05.2392888Z fi 2024-06-26T07:07:05.2393197Z  2024-06-26T07:07:05.2393690Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-06-26T07:07:05.2394352Z cat /etc/hosts 2024-06-26T07:07:05.2407749Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:05.2408358Z env: 2024-06-26T07:07:05.2408679Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:05.2409094Z ##[endgroup] 2024-06-26T07:07:05.2445361Z + PT_DOMAIN=download.pytorch.org 2024-06-26T07:07:05.2452405Z ++ dig -4 +short download.pytorch.org 2024-06-26T07:07:05.2453473Z ++ tail -n1 2024-06-26T07:07:05.2671402Z + RESOLVED_IP=13.249.85.22 2024-06-26T07:07:05.2672140Z + '[' -z 13.249.85.22 ']' 2024-06-26T07:07:05.2672679Z + grep -r download.pytorch.org /etc/hosts 2024-06-26T07:07:05.2684512Z 13.249.85.22 download.pytorch.org 2024-06-26T07:07:05.2685656Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-06-26T07:07:05.2804328Z + echo '13.249.85.22 download.pytorch.org' 2024-06-26T07:07:05.2805368Z + sudo tee -a /etc/hosts 2024-06-26T07:07:05.2883110Z 13.249.85.22 download.pytorch.org 2024-06-26T07:07:05.2891388Z + cat /etc/hosts 2024-06-26T07:07:05.2899967Z 127.0.0.1 localhost 2024-06-26T07:07:05.2900308Z 2024-06-26T07:07:05.2900813Z # The following lines are desirable for IPv6 capable hosts 2024-06-26T07:07:05.2916663Z ::1 ip6-localhost ip6-loopback 2024-06-26T07:07:05.2917447Z fe00::0 ip6-localnet 2024-06-26T07:07:05.2917902Z ff00::0 ip6-mcastprefix 2024-06-26T07:07:05.2918342Z ff02::1 ip6-allnodes 2024-06-26T07:07:05.2918760Z ff02::2 ip6-allrouters 2024-06-26T07:07:05.2919190Z ff02::3 ip6-allhosts 2024-06-26T07:07:05.2919658Z 169.254.169.254 metadata.google.internal metadata 2024-06-26T07:07:05.2920249Z 13.249.85.22 download.pytorch.org 2024-06-26T07:07:05.3120678Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-06-26T07:07:05.3121716Z with: 2024-06-26T07:07:05.3123126Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:05.3124784Z docker-build-dir: .ci/docker 2024-06-26T07:07:05.3125233Z working-directory: . 2024-06-26T07:07:05.3125800Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:05.3126429Z force-push: false 2024-06-26T07:07:05.3126776Z env: 2024-06-26T07:07:05.3127084Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:05.3127481Z ##[endgroup] 2024-06-26T07:07:05.3150103Z ##[group]Run set -ex 2024-06-26T07:07:05.3150545Z set -ex 2024-06-26T07:07:05.3150870Z  2024-06-26T07:07:05.3151551Z # If the docker build directory or the build script doesn't exist, the action will 2024-06-26T07:07:05.3152805Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-06-26T07:07:05.3153869Z # job could then download the pre-built image as usual 2024-06-26T07:07:05.3154799Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-06-26T07:07:05.3155629Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:05.3156404Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:05.3157088Z  2024-06-26T07:07:05.3157688Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-06-26T07:07:05.3158452Z  exit 0 2024-06-26T07:07:05.3158799Z else 2024-06-26T07:07:05.3159222Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:05.3159744Z fi 2024-06-26T07:07:05.3160058Z  2024-06-26T07:07:05.3160624Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-06-26T07:07:05.3161733Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-06-26T07:07:05.3162683Z  # use it as it is, but first let's extract the tag 2024-06-26T07:07:05.3163546Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-06-26T07:07:05.3164475Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:05.3165328Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:05.3165996Z else 2024-06-26T07:07:05.3166505Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-06-26T07:07:05.3167311Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:05.3168430Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:05.3169350Z fi 2024-06-26T07:07:05.3182412Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:05.3183030Z env: 2024-06-26T07:07:05.3183345Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:05.3183917Z REPO_NAME: pytorch 2024-06-26T07:07:05.3185544Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:05.3187116Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:07:05.3187721Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:05.3188384Z ##[endgroup] 2024-06-26T07:07:05.3225846Z + [[ ! -d .ci/docker ]] 2024-06-26T07:07:05.3226412Z + [[ ! -f .ci/docker/build.sh ]] 2024-06-26T07:07:05.3226858Z + echo skip=false 2024-06-26T07:07:05.3229726Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-06-26T07:07:05.3235829Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:05.3237478Z ++ awk -F '[:,]' '{print $2}' 2024-06-26T07:07:05.3260969Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:05.3261941Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:05.3265398Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:05.3302417Z ##[group]Run set +e 2024-06-26T07:07:05.3302869Z set +e 2024-06-26T07:07:05.3303215Z set -x 2024-06-26T07:07:05.3303533Z  2024-06-26T07:07:05.3303888Z login() { 2024-06-26T07:07:05.3305216Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:07:05.3306111Z } 2024-06-26T07:07:05.3306428Z  2024-06-26T07:07:05.3306742Z retry () { 2024-06-26T07:07:05.3307202Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:07:05.3307735Z } 2024-06-26T07:07:05.3308032Z  2024-06-26T07:07:05.3308394Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:07:05.3308945Z  2024-06-26T07:07:05.3309503Z # Check if image already exists, if it does then skip building it 2024-06-26T07:07:05.3310382Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-06-26T07:07:05.3310989Z  exit 0 2024-06-26T07:07:05.3311348Z fi 2024-06-26T07:07:05.3311666Z  2024-06-26T07:07:05.3312841Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-06-26T07:07:05.3313998Z # be empty. The default action would be to continue rebuild the image 2024-06-26T07:07:05.3314938Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-06-26T07:07:05.3315774Z  # if we're on the base branch then use the parent commit 2024-06-26T07:07:05.3316482Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-06-26T07:07:05.3316999Z else 2024-06-26T07:07:05.3317556Z  # otherwise we're on a PR, so use the most recent base commit 2024-06-26T07:07:05.3318406Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-06-26T07:07:05.3319010Z fi 2024-06-26T07:07:05.3319332Z  2024-06-26T07:07:05.3319695Z if [[ -z "${MERGE_BASE}" ]]; then 2024-06-26T07:07:05.3320288Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:05.3320835Z  2024-06-26T07:07:05.3321665Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-06-26T07:07:05.3322622Z  exit 0 2024-06-26T07:07:05.3322953Z fi 2024-06-26T07:07:05.3323270Z  2024-06-26T07:07:05.3323848Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-06-26T07:07:05.3325080Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-06-26T07:07:05.3326085Z  exit 1 2024-06-26T07:07:05.3326415Z fi 2024-06-26T07:07:05.3326727Z  2024-06-26T07:07:05.3327309Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-06-26T07:07:05.3328493Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-06-26T07:07:05.3329559Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-06-26T07:07:05.3330770Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-06-26T07:07:05.3332175Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-06-26T07:07:05.3333260Z fi 2024-06-26T07:07:05.3333763Z  2024-06-26T07:07:05.3334178Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:05.3347694Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:05.3348289Z env: 2024-06-26T07:07:05.3348613Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:05.3349047Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:07:05.3349585Z BASE_REVISION: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:07:05.3351232Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:05.3352856Z DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:05.3353592Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:05.3354288Z ##[endgroup] 2024-06-26T07:07:05.3409100Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:05.3409938Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:05.3412535Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:07:05.3414404Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:06.4993883Z WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json. 2024-06-26T07:07:06.4995509Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:06.4997647Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:06.4998823Z 2024-06-26T07:07:06.4999429Z Login Succeeded 2024-06-26T07:07:06.5020694Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:07.1747812Z { 2024-06-26T07:07:07.1748572Z "schemaVersion": 2, 2024-06-26T07:07:07.1749489Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-06-26T07:07:07.1750203Z "config": { 2024-06-26T07:07:07.1750768Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-06-26T07:07:07.1751442Z "size": 48681, 2024-06-26T07:07:07.1752120Z "digest": "sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd" 2024-06-26T07:07:07.1753240Z }, 2024-06-26T07:07:07.1753540Z "layers": [ 2024-06-26T07:07:07.1754037Z { 2024-06-26T07:07:07.1755020Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1756244Z "size": 28580681, 2024-06-26T07:07:07.1757475Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-06-26T07:07:07.1758617Z }, 2024-06-26T07:07:07.1758902Z { 2024-06-26T07:07:07.1759410Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1760096Z "size": 7943451, 2024-06-26T07:07:07.1760756Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-06-26T07:07:07.1761502Z }, 2024-06-26T07:07:07.1761792Z { 2024-06-26T07:07:07.1762542Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1763644Z "size": 55728572, 2024-06-26T07:07:07.1764900Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-06-26T07:07:07.1766390Z }, 2024-06-26T07:07:07.1766825Z { 2024-06-26T07:07:07.1767679Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1768919Z "size": 186, 2024-06-26T07:07:07.1770110Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-06-26T07:07:07.1771614Z }, 2024-06-26T07:07:07.1772117Z { 2024-06-26T07:07:07.1772904Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1773959Z "size": 6886, 2024-06-26T07:07:07.1775157Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-06-26T07:07:07.1776241Z }, 2024-06-26T07:07:07.1776974Z { 2024-06-26T07:07:07.1777514Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1778392Z "size": 1329001756, 2024-06-26T07:07:07.1779111Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-06-26T07:07:07.1779883Z }, 2024-06-26T07:07:07.1780161Z { 2024-06-26T07:07:07.1780894Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1781606Z "size": 62501, 2024-06-26T07:07:07.1782264Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-06-26T07:07:07.1783034Z }, 2024-06-26T07:07:07.1783311Z { 2024-06-26T07:07:07.1783830Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1784825Z "size": 1684, 2024-06-26T07:07:07.1785479Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-06-26T07:07:07.1786231Z }, 2024-06-26T07:07:07.1786493Z { 2024-06-26T07:07:07.1787023Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1787713Z "size": 1523, 2024-06-26T07:07:07.1788400Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-06-26T07:07:07.1789178Z }, 2024-06-26T07:07:07.1789451Z { 2024-06-26T07:07:07.1789969Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1790656Z "size": 2528295403, 2024-06-26T07:07:07.1791341Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-06-26T07:07:07.1792110Z }, 2024-06-26T07:07:07.1792384Z { 2024-06-26T07:07:07.1792891Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1793601Z "size": 86016, 2024-06-26T07:07:07.1794326Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-06-26T07:07:07.1795105Z }, 2024-06-26T07:07:07.1795366Z { 2024-06-26T07:07:07.1795897Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1796584Z "size": 1823, 2024-06-26T07:07:07.1797236Z "digest": "sha256:309b872da805fcdf74137a887464876fa0615dc8b24199307119eb65fc2bfb49" 2024-06-26T07:07:07.1797971Z }, 2024-06-26T07:07:07.1798245Z { 2024-06-26T07:07:07.1798761Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1799444Z "size": 246737542, 2024-06-26T07:07:07.1800100Z "digest": "sha256:e82932cb002c06a782077b8e5965f8733eb8563406a23cab260d9d8c4a994f7d" 2024-06-26T07:07:07.1800842Z }, 2024-06-26T07:07:07.1801117Z { 2024-06-26T07:07:07.1801622Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1802308Z "size": 545, 2024-06-26T07:07:07.1802974Z "digest": "sha256:ca9dd73e3e9378225ebdcfd12199ef44649e34fe55003cfbc8afe2e8a4527c5b" 2024-06-26T07:07:07.1803744Z }, 2024-06-26T07:07:07.1804008Z { 2024-06-26T07:07:07.1804571Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1805257Z "size": 1283, 2024-06-26T07:07:07.1805909Z "digest": "sha256:16d76a79d5d6616e228537785ec83a869f68a402520bbc6e826201621ca6cb98" 2024-06-26T07:07:07.1806633Z }, 2024-06-26T07:07:07.1806907Z { 2024-06-26T07:07:07.1807422Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1808105Z "size": 485, 2024-06-26T07:07:07.1808765Z "digest": "sha256:4bdf41c9ecbe1651e0986ccbea4663a0556af27b7145a679cfa45c35dc96b891" 2024-06-26T07:07:07.1809542Z }, 2024-06-26T07:07:07.1809816Z { 2024-06-26T07:07:07.1810316Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1810997Z "size": 91710937, 2024-06-26T07:07:07.1811702Z "digest": "sha256:f4863801e30cdbcd1d4e6cb1cbbcde7dbcaf6ae317ca80429295741ce60ac31d" 2024-06-26T07:07:07.1812491Z }, 2024-06-26T07:07:07.1812750Z { 2024-06-26T07:07:07.1813266Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1813946Z "size": 3233, 2024-06-26T07:07:07.1814909Z "digest": "sha256:d37dd9ed8948a0d02e1198bbedadbca36477a0980dd553196385aab0c025581d" 2024-06-26T07:07:07.1815831Z }, 2024-06-26T07:07:07.1816106Z { 2024-06-26T07:07:07.1816625Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1817294Z "size": 1905, 2024-06-26T07:07:07.1817972Z "digest": "sha256:cf2f34f225dbed17c1a9a8b9ea4ad030aba797774a716dc0a82fe212324aa751" 2024-06-26T07:07:07.1818754Z }, 2024-06-26T07:07:07.1819026Z { 2024-06-26T07:07:07.1819530Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1820214Z "size": 702, 2024-06-26T07:07:07.1820880Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:07.1821652Z }, 2024-06-26T07:07:07.1821909Z { 2024-06-26T07:07:07.1822421Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1823102Z "size": 2784789002, 2024-06-26T07:07:07.1823827Z "digest": "sha256:3d914950f8addfc4d657a2365c76e1143eb80b710cd3f6e9ae05640217fc6347" 2024-06-26T07:07:07.1824936Z }, 2024-06-26T07:07:07.1825215Z { 2024-06-26T07:07:07.1825736Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1826402Z "size": 381, 2024-06-26T07:07:07.1827059Z "digest": "sha256:96d0e2032184c57e62a652aaffcf0241f0d0c7a23e894ca0ad192c4a64a36583" 2024-06-26T07:07:07.1827963Z + exit 0 2024-06-26T07:07:07.1828252Z }, 2024-06-26T07:07:07.1828513Z { 2024-06-26T07:07:07.1829027Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1829703Z "size": 11879, 2024-06-26T07:07:07.1830364Z "digest": "sha256:06e12ce59790cb8b091e5d619c2a62a28329008ee184a8b9a562e3db12471f30" 2024-06-26T07:07:07.1831108Z }, 2024-06-26T07:07:07.1831380Z { 2024-06-26T07:07:07.1831897Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1832572Z "size": 803, 2024-06-26T07:07:07.1833232Z "digest": "sha256:ed712f63cde7070bdec2a6179c4c35e752c6991ddbc69fe9f2724a9903250ceb" 2024-06-26T07:07:07.1834053Z }, 2024-06-26T07:07:07.1834365Z { 2024-06-26T07:07:07.1834867Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1835615Z "size": 106, 2024-06-26T07:07:07.1836268Z "digest": "sha256:a97a71ef8a4e66234dfc2098834f8395ac5630240e60164d1a251fdb0115ba85" 2024-06-26T07:07:07.1837024Z }, 2024-06-26T07:07:07.1837285Z { 2024-06-26T07:07:07.1837805Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1838482Z "size": 503, 2024-06-26T07:07:07.1839134Z "digest": "sha256:25cd0c482ad612f936b35b4f220c580e16be982226aac8801a6a849a6bfdc221" 2024-06-26T07:07:07.1839873Z }, 2024-06-26T07:07:07.1840147Z { 2024-06-26T07:07:07.1840659Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1841326Z "size": 121477177, 2024-06-26T07:07:07.1842020Z "digest": "sha256:9bf94bc456584365fda21d0ed83b7cc690b08a71727b71c4c1e667e15028a386" 2024-06-26T07:07:07.1842789Z }, 2024-06-26T07:07:07.1843059Z { 2024-06-26T07:07:07.1843566Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1844293Z "size": 109, 2024-06-26T07:07:07.1844949Z "digest": "sha256:de45a14c9b227e4b0c9d45d42223b50a71c03522ddd9e2ca0d9517835206c823" 2024-06-26T07:07:07.1845773Z }, 2024-06-26T07:07:07.1846035Z { 2024-06-26T07:07:07.1846552Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1847240Z "size": 491, 2024-06-26T07:07:07.1847893Z "digest": "sha256:e99241e77209b0b35a0fb59a61c0821fc161190178af7e4af45e6eeaa043375e" 2024-06-26T07:07:07.1848627Z }, 2024-06-26T07:07:07.1848898Z { 2024-06-26T07:07:07.1849415Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1850077Z "size": 291, 2024-06-26T07:07:07.1850753Z "digest": "sha256:35e8f4e2a42f9df0c969dedd2199cf71719cf5ecfa676dfa014bbdeb4fdb295f" 2024-06-26T07:07:07.1851752Z }, 2024-06-26T07:07:07.1852158Z { 2024-06-26T07:07:07.1852663Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1853343Z "size": 101, 2024-06-26T07:07:07.1853994Z "digest": "sha256:a823ffa4c541394b52953391dc0a6867e26c37fe4a71fa022a2cc3f07f0ba70d" 2024-06-26T07:07:07.1854800Z }, 2024-06-26T07:07:07.1878349Z { 2024-06-26T07:07:07.1879044Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1879753Z "size": 1473, 2024-06-26T07:07:07.1880411Z "digest": "sha256:44ba272eae2064603605a6238b3f35cc6578859d660068e702a047a9cd8ab2fd" 2024-06-26T07:07:07.1881171Z }, 2024-06-26T07:07:07.1881436Z { 2024-06-26T07:07:07.1881956Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1882643Z "size": 446710781, 2024-06-26T07:07:07.1883317Z "digest": "sha256:da02d278e5240c285bc855ce4e42cd6121d5b903787f5c14f03e575ad1268650" 2024-06-26T07:07:07.1884074Z }, 2024-06-26T07:07:07.1884426Z { 2024-06-26T07:07:07.1884971Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1885630Z "size": 161, 2024-06-26T07:07:07.1886253Z "digest": "sha256:73a0ce681c295135456e190578733273276b38b4dd5babbb39298f384d43b1c6" 2024-06-26T07:07:07.1886977Z }, 2024-06-26T07:07:07.1887248Z { 2024-06-26T07:07:07.1887755Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1888438Z "size": 564, 2024-06-26T07:07:07.1889091Z "digest": "sha256:49737bd5ae709dbf7956e21fc8b05ba009e9e6c9b130b0126a73bd3d3f5efc78" 2024-06-26T07:07:07.1889844Z }, 2024-06-26T07:07:07.1890114Z { 2024-06-26T07:07:07.1890624Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1891298Z "size": 35874600, 2024-06-26T07:07:07.1891971Z "digest": "sha256:9cd4c745f8e4f9d5c9bb8050923575afad21f5b7852975a4a1981bc9eff282c5" 2024-06-26T07:07:07.1892724Z }, 2024-06-26T07:07:07.1892988Z { 2024-06-26T07:07:07.1893510Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1894222Z "size": 104, 2024-06-26T07:07:07.1894861Z "digest": "sha256:b2950a0365345d4c17a48627c905fd8930b3b3154c727f0526eb9ca547102bb7" 2024-06-26T07:07:07.1895605Z }, 2024-06-26T07:07:07.1895876Z { 2024-06-26T07:07:07.1896375Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1897046Z "size": 425, 2024-06-26T07:07:07.1897688Z "digest": "sha256:25e19d81d9fc081962ff4cbc09e098f4035419822712c19a410f5282300dfb9e" 2024-06-26T07:07:07.1898415Z }, 2024-06-26T07:07:07.1898687Z { 2024-06-26T07:07:07.1899201Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1899874Z "size": 20262084, 2024-06-26T07:07:07.1900547Z "digest": "sha256:60156b40fcbec2c25f0170b0a82fa8cd9303e0daae8bb1811fd0fc6562e88e8e" 2024-06-26T07:07:07.1901451Z }, 2024-06-26T07:07:07.1901713Z { 2024-06-26T07:07:07.1902226Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1902886Z "size": 439, 2024-06-26T07:07:07.1903522Z "digest": "sha256:7a868239227fd19b30a19927bbc8f058d247403fe5b343c55f4b4b8987918911" 2024-06-26T07:07:07.1904605Z }, 2024-06-26T07:07:07.1904868Z { 2024-06-26T07:07:07.1905392Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1906072Z "size": 702, 2024-06-26T07:07:07.1906739Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:07.1907492Z }, 2024-06-26T07:07:07.1907764Z { 2024-06-26T07:07:07.1908274Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1908957Z "size": 143, 2024-06-26T07:07:07.1909594Z "digest": "sha256:83911d7c7ecf11b5932bb7449975c52e7f737a5b867691935e1ccb85ceb25b52" 2024-06-26T07:07:07.1910355Z }, 2024-06-26T07:07:07.1910630Z { 2024-06-26T07:07:07.1911126Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1912089Z "size": 135, 2024-06-26T07:07:07.1912896Z "digest": "sha256:b3c5d66c679f58c71febdd502889bd8cd3d36e5001754cdf1e30eabe21135cce" 2024-06-26T07:07:07.1913672Z }, 2024-06-26T07:07:07.1913931Z { 2024-06-26T07:07:07.1914513Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1915202Z "size": 4911495247, 2024-06-26T07:07:07.1915881Z "digest": "sha256:84b2c7a899820f6acdb22148c5e8346d8b26c33269fd01842c9573515251e1e1" 2024-06-26T07:07:07.1916623Z }, 2024-06-26T07:07:07.1916895Z { 2024-06-26T07:07:07.1917410Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1918094Z "size": 190, 2024-06-26T07:07:07.1918756Z "digest": "sha256:c1068d03a811d8dead5da9e82fdc8869205fc4ee9cb92d4f7a4acfbb5ba83950" 2024-06-26T07:07:07.1919540Z }, 2024-06-26T07:07:07.1919814Z { 2024-06-26T07:07:07.1920319Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1921007Z "size": 565, 2024-06-26T07:07:07.1921662Z "digest": "sha256:671b4a301d81074fca17d36851f81801552c4ecb2f90ff3412aa9ab6a1a91371" 2024-06-26T07:07:07.1922415Z }, 2024-06-26T07:07:07.1922678Z { 2024-06-26T07:07:07.1923195Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1923912Z "size": 43163095, 2024-06-26T07:07:07.1924641Z "digest": "sha256:67760cf5ec46ab58d3ca98637fd9b26cb47a67446bfd0db176fe5e023606aa0d" 2024-06-26T07:07:07.1925396Z }, 2024-06-26T07:07:07.1925671Z { 2024-06-26T07:07:07.1926184Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1926862Z "size": 106, 2024-06-26T07:07:07.1927510Z "digest": "sha256:ddc9740bcea2b301e99b50525059aea8d9c7121a1c8eab566561a8968438a8c2" 2024-06-26T07:07:07.1928277Z }, 2024-06-26T07:07:07.1928554Z { 2024-06-26T07:07:07.1929055Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1929732Z "size": 1213, 2024-06-26T07:07:07.1930387Z "digest": "sha256:fb2e977685609fcc01de170ba7f50a573e73e6520e47253717e92a28952c5ee8" 2024-06-26T07:07:07.1931141Z }, 2024-06-26T07:07:07.1931405Z { 2024-06-26T07:07:07.1931922Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1932642Z "size": 702, 2024-06-26T07:07:07.1933307Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:07.1934099Z }, 2024-06-26T07:07:07.1934403Z { 2024-06-26T07:07:07.1934925Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1935604Z "size": 138, 2024-06-26T07:07:07.1936259Z "digest": "sha256:b599c5a8c7cb461cbd7b3c98b99960b8eb04506e05e79db1cc0cfadb14a47e91" 2024-06-26T07:07:07.1937033Z }, 2024-06-26T07:07:07.1937307Z { 2024-06-26T07:07:07.1937807Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1938481Z "size": 120, 2024-06-26T07:07:07.1939159Z "digest": "sha256:e08e88c1366827455ee7ddbf92b6ae615ceb3cbd3fb11cd03ddd31bd3fa94a9c" 2024-06-26T07:07:07.1939932Z }, 2024-06-26T07:07:07.1940190Z { 2024-06-26T07:07:07.1940705Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1941385Z "size": 1904157495, 2024-06-26T07:07:07.1942086Z "digest": "sha256:12ba7431ff38de8cf6af3052a2a52e79f4e712b8c99f5bafde3c20e63b1f570f" 2024-06-26T07:07:07.1942838Z }, 2024-06-26T07:07:07.1943108Z { 2024-06-26T07:07:07.1943621Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1944651Z "size": 172, 2024-06-26T07:07:07.1945320Z "digest": "sha256:cf7c2cb896cba6a594028cc0fd246e127a2f4481d48905a3aa12c5f5872332d4" 2024-06-26T07:07:07.1946095Z }, 2024-06-26T07:07:07.1946365Z { 2024-06-26T07:07:07.1946872Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1947543Z "size": 908, 2024-06-26T07:07:07.1948216Z "digest": "sha256:3a778ddfac2af5b3a11f49ddf2f3f1ad1731f74ba691f434ab69966d1e2a189f" 2024-06-26T07:07:07.1949235Z }, 2024-06-26T07:07:07.1949624Z { 2024-06-26T07:07:07.1950140Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1950822Z "size": 702, 2024-06-26T07:07:07.1951483Z "digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4" 2024-06-26T07:07:07.1952234Z }, 2024-06-26T07:07:07.1952508Z { 2024-06-26T07:07:07.1953018Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1953683Z "size": 134, 2024-06-26T07:07:07.1954381Z "digest": "sha256:236c8f0245f4b40616a4322eaf1e1b3e178d5990db1fcaba39622ea4e79c06a1" 2024-06-26T07:07:07.1955144Z }, 2024-06-26T07:07:07.1955417Z { 2024-06-26T07:07:07.1955918Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1956592Z "size": 32, 2024-06-26T07:07:07.1957247Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:07.1958011Z }, 2024-06-26T07:07:07.1958268Z { 2024-06-26T07:07:07.1958791Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1959484Z "size": 156, 2024-06-26T07:07:07.1960146Z "digest": "sha256:dda1ce302b9a9df0c8f1145d2e3eaa03c96909bb45a792451f1fcb555b06738d" 2024-06-26T07:07:07.1960913Z }, 2024-06-26T07:07:07.1961184Z { 2024-06-26T07:07:07.1961694Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1962359Z "size": 1841, 2024-06-26T07:07:07.1963026Z "digest": "sha256:e797fe86caef19319e03f8dca92cf96838ab3bb3644f4447c6dfc04230118afa" 2024-06-26T07:07:07.1963792Z }, 2024-06-26T07:07:07.1964061Z { 2024-06-26T07:07:07.1964623Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1965300Z "size": 7529784, 2024-06-26T07:07:07.1965971Z "digest": "sha256:967190076fc0ab2bb2fc0d74e58edd3d662a537d702ff3f8b6e876230154a16f" 2024-06-26T07:07:07.1966729Z }, 2024-06-26T07:07:07.1966990Z { 2024-06-26T07:07:07.1967511Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1968198Z "size": 165, 2024-06-26T07:07:07.1968835Z "digest": "sha256:4fb0a83f06855534ef82b80dcf7c685303edb2bcbd446196066dc2ad2dd71281" 2024-06-26T07:07:07.1969589Z }, 2024-06-26T07:07:07.1969857Z { 2024-06-26T07:07:07.1970369Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1971031Z "size": 7944, 2024-06-26T07:07:07.1971694Z "digest": "sha256:68af2ec4e30a33f26947ece9f3a31b7d3b04c2bca5865d4b48618457832a1da2" 2024-06-26T07:07:07.1972449Z }, 2024-06-26T07:07:07.1972719Z { 2024-06-26T07:07:07.1973217Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1973884Z "size": 8070, 2024-06-26T07:07:07.1974574Z "digest": "sha256:cc91933e985a044c6df53caa15488d2285896f5e991f38ec068c253ceec53f14" 2024-06-26T07:07:07.1975317Z }, 2024-06-26T07:07:07.1975593Z { 2024-06-26T07:07:07.1976106Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1976790Z "size": 303, 2024-06-26T07:07:07.1977438Z "digest": "sha256:9f3187916e7dcad33cc572f8f0e949d2f507b887f99cfd6acefd14826b44960d" 2024-06-26T07:07:07.1978201Z }, 2024-06-26T07:07:07.1978471Z { 2024-06-26T07:07:07.1978984Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1979651Z "size": 7629071, 2024-06-26T07:07:07.1980329Z "digest": "sha256:52608399485738d10d779254946c9e1d536010dd45593c69bf1eb9f62f85dc24" 2024-06-26T07:07:07.1981060Z }, 2024-06-26T07:07:07.1981342Z { 2024-06-26T07:07:07.1981846Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1982521Z "size": 108, 2024-06-26T07:07:07.1983208Z "digest": "sha256:05a6f6dcc1c00bac6660923a22adc7d7c34416718f8c89daf9b3351d9365f694" 2024-06-26T07:07:07.1983963Z }, 2024-06-26T07:07:07.1984572Z { 2024-06-26T07:07:07.1985093Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1985976Z "size": 54145775, 2024-06-26T07:07:07.1986769Z "digest": "sha256:6bc9dc8315b09b27c976bfba9700767ba412d72fd0528b18bbe8f8e62005ec22" 2024-06-26T07:07:07.1987531Z }, 2024-06-26T07:07:07.1987806Z { 2024-06-26T07:07:07.1988327Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1988997Z "size": 474, 2024-06-26T07:07:07.1989653Z "digest": "sha256:1b0e27c35f07b3fbf0aeb548d1949bf1d60a420136da21789d94d196c891d981" 2024-06-26T07:07:07.1990413Z }, 2024-06-26T07:07:07.1990672Z { 2024-06-26T07:07:07.1991187Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1991863Z "size": 1374858747, 2024-06-26T07:07:07.1992532Z "digest": "sha256:95237e7311a1c5ce6937db7196c866632f44e8788514a5c0eb4a3f04da4017e6" 2024-06-26T07:07:07.1993261Z }, 2024-06-26T07:07:07.1993532Z { 2024-06-26T07:07:07.1994045Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1994771Z "size": 106, 2024-06-26T07:07:07.1995459Z "digest": "sha256:789eff6c51aada70bf75eb1acafed63c0d839b768638001500616ff6e53b854a" 2024-06-26T07:07:07.1996247Z }, 2024-06-26T07:07:07.1996518Z { 2024-06-26T07:07:07.1997036Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.1997698Z "size": 558, 2024-06-26T07:07:07.1998368Z "digest": "sha256:de0f8a8c45c6077b3c5ba3b84d7f07de98353fd618dad9a1e2f368ffe00493c7" 2024-06-26T07:07:07.1999148Z }, 2024-06-26T07:07:07.1999410Z { 2024-06-26T07:07:07.1999925Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.2000609Z "size": 46248513, 2024-06-26T07:07:07.2001270Z "digest": "sha256:2060028f74888d218bebdd10d723f66c33ee1c708935c4ffd4654d34912ac4dc" 2024-06-26T07:07:07.2002005Z }, 2024-06-26T07:07:07.2002279Z { 2024-06-26T07:07:07.2002792Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.2003467Z "size": 111, 2024-06-26T07:07:07.2004115Z "digest": "sha256:e412c7f3b2ded440992955bd92512345cbece8a8894cd3592f714ebc758ad1a0" 2024-06-26T07:07:07.2004926Z }, 2024-06-26T07:07:07.2005195Z { 2024-06-26T07:07:07.2005700Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.2006371Z "size": 32, 2024-06-26T07:07:07.2007030Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:07.2007797Z }, 2024-06-26T07:07:07.2008054Z { 2024-06-26T07:07:07.2009133Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.2009812Z "size": 32, 2024-06-26T07:07:07.2010471Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:07.2011226Z }, 2024-06-26T07:07:07.2011503Z { 2024-06-26T07:07:07.2012017Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.2012692Z "size": 32, 2024-06-26T07:07:07.2013392Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:07.2014163Z }, 2024-06-26T07:07:07.2014465Z { 2024-06-26T07:07:07.2014969Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:07.2015637Z "size": 32, 2024-06-26T07:07:07.2016293Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:07.2017059Z } 2024-06-26T07:07:07.2017320Z ] 2024-06-26T07:07:07.2017588Z } 2024-06-26T07:07:07.2153945Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:07:07.2154637Z tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:07:07.2155297Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-06-26T07:07:07.2169253Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:07.2169852Z env: 2024-06-26T07:07:07.2170168Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:07.2171672Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:07.2173428Z ##[endgroup] 2024-06-26T07:07:07.2214391Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:07.2280645Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-06-26T07:07:07.2281368Z with: 2024-06-26T07:07:07.2282755Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:07.2284535Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:07.2285150Z env: 2024-06-26T07:07:07.2285460Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:07.2285858Z ##[endgroup] 2024-06-26T07:07:07.2307065Z ##[group]Run set -x 2024-06-26T07:07:07.2307481Z set -x 2024-06-26T07:07:07.2307832Z set +e 2024-06-26T07:07:07.2308172Z  2024-06-26T07:07:07.2308475Z login() { 2024-06-26T07:07:07.2309299Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:07:07.2310196Z } 2024-06-26T07:07:07.2310510Z  2024-06-26T07:07:07.2310870Z retry () { 2024-06-26T07:07:07.2311326Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:07:07.2311840Z } 2024-06-26T07:07:07.2312145Z  2024-06-26T07:07:07.2312504Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:07:07.2312986Z  2024-06-26T07:07:07.2313288Z set -e 2024-06-26T07:07:07.2313851Z # ignore output since only exit code is used for conditional 2024-06-26T07:07:07.2315005Z # only pull docker image if it's not available locally 2024-06-26T07:07:07.2315949Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-06-26T07:07:07.2316813Z  retry docker pull "${DOCKER_IMAGE}" 2024-06-26T07:07:07.2317342Z fi 2024-06-26T07:07:07.2331538Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:07.2332147Z env: 2024-06-26T07:07:07.2332452Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:07.2333999Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:07.2336139Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:07.2336777Z ##[endgroup] 2024-06-26T07:07:07.2379604Z + set +e 2024-06-26T07:07:07.2380438Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:07.2381233Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:07.2383507Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:07:07.2385375Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:08.3816626Z WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json. 2024-06-26T07:07:08.3818214Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:08.3819943Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:08.3820636Z 2024-06-26T07:07:08.3820787Z Login Succeeded 2024-06-26T07:07:08.3843041Z + set -e 2024-06-26T07:07:08.3844956Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:08.4062055Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:08.4065207Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:08.9692396Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-06-26T07:07:08.9696315Z 7a2c55901189: Pulling fs layer 2024-06-26T07:07:08.9697062Z 224fe954d725: Pulling fs layer 2024-06-26T07:07:08.9698361Z 75722010b82e: Pulling fs layer 2024-06-26T07:07:08.9699209Z d527cbbb87e3: Pulling fs layer 2024-06-26T07:07:08.9699990Z b57676e46aee: Pulling fs layer 2024-06-26T07:07:08.9700833Z a8c1e85b5e14: Pulling fs layer 2024-06-26T07:07:08.9701846Z a41a8d1c11c8: Pulling fs layer 2024-06-26T07:07:08.9702602Z 0c1227890755: Pulling fs layer 2024-06-26T07:07:08.9703302Z d8d1234baab3: Pulling fs layer 2024-06-26T07:07:08.9704339Z 7ed32bc8e469: Pulling fs layer 2024-06-26T07:07:08.9705017Z ec1e7978c1fe: Pulling fs layer 2024-06-26T07:07:08.9705458Z 309b872da805: Pulling fs layer 2024-06-26T07:07:08.9705896Z e82932cb002c: Pulling fs layer 2024-06-26T07:07:08.9706343Z ca9dd73e3e93: Pulling fs layer 2024-06-26T07:07:08.9706772Z 16d76a79d5d6: Pulling fs layer 2024-06-26T07:07:08.9707223Z 4bdf41c9ecbe: Pulling fs layer 2024-06-26T07:07:08.9707680Z f4863801e30c: Pulling fs layer 2024-06-26T07:07:08.9708117Z d37dd9ed8948: Pulling fs layer 2024-06-26T07:07:08.9708543Z cf2f34f225db: Pulling fs layer 2024-06-26T07:07:08.9708999Z df546d7b3e34: Pulling fs layer 2024-06-26T07:07:08.9709504Z d8d1234baab3: Waiting 2024-06-26T07:07:08.9709880Z 3d914950f8ad: Pulling fs layer 2024-06-26T07:07:08.9710300Z 7ed32bc8e469: Waiting 2024-06-26T07:07:08.9710668Z a41a8d1c11c8: Waiting 2024-06-26T07:07:08.9711053Z 96d0e2032184: Pulling fs layer 2024-06-26T07:07:08.9711477Z 06e12ce59790: Pulling fs layer 2024-06-26T07:07:08.9711982Z ed712f63cde7: Pulling fs layer 2024-06-26T07:07:08.9712426Z a97a71ef8a4e: Pulling fs layer 2024-06-26T07:07:08.9712845Z 0c1227890755: Waiting 2024-06-26T07:07:08.9713227Z 25cd0c482ad6: Pulling fs layer 2024-06-26T07:07:08.9713649Z 9bf94bc45658: Pulling fs layer 2024-06-26T07:07:08.9714084Z de45a14c9b22: Pulling fs layer 2024-06-26T07:07:08.9714521Z e99241e77209: Pulling fs layer 2024-06-26T07:07:08.9714928Z ca9dd73e3e93: Waiting 2024-06-26T07:07:08.9715322Z 35e8f4e2a42f: Pulling fs layer 2024-06-26T07:07:08.9715762Z a823ffa4c541: Pulling fs layer 2024-06-26T07:07:08.9716184Z 309b872da805: Waiting 2024-06-26T07:07:08.9716560Z 44ba272eae20: Pulling fs layer 2024-06-26T07:07:08.9716984Z a97a71ef8a4e: Waiting 2024-06-26T07:07:08.9717377Z da02d278e524: Pulling fs layer 2024-06-26T07:07:08.9717802Z e82932cb002c: Waiting 2024-06-26T07:07:08.9718155Z 25cd0c482ad6: Waiting 2024-06-26T07:07:08.9718542Z 73a0ce681c29: Pulling fs layer 2024-06-26T07:07:08.9718962Z b57676e46aee: Waiting 2024-06-26T07:07:08.9719373Z 49737bd5ae70: Pulling fs layer 2024-06-26T07:07:08.9719812Z 9cd4c745f8e4: Pulling fs layer 2024-06-26T07:07:08.9720248Z b2950a036534: Pulling fs layer 2024-06-26T07:07:08.9720682Z 25e19d81d9fc: Pulling fs layer 2024-06-26T07:07:08.9721092Z 4bdf41c9ecbe: Waiting 2024-06-26T07:07:08.9721460Z 9bf94bc45658: Waiting 2024-06-26T07:07:08.9721839Z 60156b40fcbe: Pulling fs layer 2024-06-26T07:07:08.9722262Z de45a14c9b22: Waiting 2024-06-26T07:07:08.9722634Z 7a868239227f: Pulling fs layer 2024-06-26T07:07:08.9723054Z f4863801e30c: Waiting 2024-06-26T07:07:08.9723417Z cf2f34f225db: Waiting 2024-06-26T07:07:08.9723793Z 83911d7c7ecf: Pulling fs layer 2024-06-26T07:07:08.9724343Z df546d7b3e34: Waiting 2024-06-26T07:07:08.9724707Z 96d0e2032184: Waiting 2024-06-26T07:07:08.9725090Z b3c5d66c679f: Pulling fs layer 2024-06-26T07:07:08.9725538Z a8c1e85b5e14: Waiting 2024-06-26T07:07:08.9725921Z 84b2c7a89982: Pulling fs layer 2024-06-26T07:07:08.9726348Z 3d914950f8ad: Waiting 2024-06-26T07:07:08.9726700Z a823ffa4c541: Waiting 2024-06-26T07:07:08.9727077Z c1068d03a811: Pulling fs layer 2024-06-26T07:07:08.9727515Z 671b4a301d81: Pulling fs layer 2024-06-26T07:07:08.9727955Z 67760cf5ec46: Pulling fs layer 2024-06-26T07:07:08.9728357Z 06e12ce59790: Waiting 2024-06-26T07:07:08.9728741Z ddc9740bcea2: Pulling fs layer 2024-06-26T07:07:08.9729204Z ed712f63cde7: Waiting 2024-06-26T07:07:08.9729589Z fb2e97768560: Pulling fs layer 2024-06-26T07:07:08.9729997Z 16d76a79d5d6: Waiting 2024-06-26T07:07:08.9730655Z 44ba272eae20: Waiting 2024-06-26T07:07:08.9731020Z b2950a036534: Waiting 2024-06-26T07:07:08.9731369Z 9cd4c745f8e4: Waiting 2024-06-26T07:07:08.9731905Z b599c5a8c7cb: Pulling fs layer 2024-06-26T07:07:08.9732340Z 25e19d81d9fc: Waiting 2024-06-26T07:07:08.9732727Z e08e88c13668: Pulling fs layer 2024-06-26T07:07:08.9733152Z 12ba7431ff38: Pulling fs layer 2024-06-26T07:07:08.9733596Z cf7c2cb896cb: Pulling fs layer 2024-06-26T07:07:08.9734023Z b3c5d66c679f: Waiting 2024-06-26T07:07:08.9734410Z 3a778ddfac2a: Pulling fs layer 2024-06-26T07:07:08.9734839Z 236c8f0245f4: Pulling fs layer 2024-06-26T07:07:08.9735305Z 84b2c7a89982: Waiting 2024-06-26T07:07:08.9735668Z c1068d03a811: Waiting 2024-06-26T07:07:08.9736035Z 4f4fb700ef54: Pulling fs layer 2024-06-26T07:07:08.9736484Z dda1ce302b9a: Pulling fs layer 2024-06-26T07:07:08.9736912Z da02d278e524: Waiting 2024-06-26T07:07:08.9737294Z e797fe86caef: Pulling fs layer 2024-06-26T07:07:08.9737701Z e99241e77209: Waiting 2024-06-26T07:07:08.9738072Z 35e8f4e2a42f: Waiting 2024-06-26T07:07:08.9738432Z 67760cf5ec46: Waiting 2024-06-26T07:07:08.9738780Z ddc9740bcea2: Waiting 2024-06-26T07:07:08.9739182Z 671b4a301d81: Waiting 2024-06-26T07:07:08.9739560Z 967190076fc0: Pulling fs layer 2024-06-26T07:07:08.9739999Z 4fb0a83f0685: Pulling fs layer 2024-06-26T07:07:08.9740427Z 68af2ec4e30a: Pulling fs layer 2024-06-26T07:07:08.9740868Z cc91933e985a: Pulling fs layer 2024-06-26T07:07:08.9741308Z 9f3187916e7d: Pulling fs layer 2024-06-26T07:07:08.9741741Z 526083994857: Pulling fs layer 2024-06-26T07:07:08.9742143Z cf7c2cb896cb: Waiting 2024-06-26T07:07:08.9742531Z 05a6f6dcc1c0: Pulling fs layer 2024-06-26T07:07:08.9742949Z 7a868239227f: Waiting 2024-06-26T07:07:08.9743316Z 6bc9dc8315b0: Pulling fs layer 2024-06-26T07:07:08.9743758Z 1b0e27c35f07: Pulling fs layer 2024-06-26T07:07:08.9744494Z 3a778ddfac2a: Waiting 2024-06-26T07:07:08.9744870Z 83911d7c7ecf: Waiting 2024-06-26T07:07:08.9745231Z 95237e7311a1: Pulling fs layer 2024-06-26T07:07:08.9745677Z 789eff6c51aa: Pulling fs layer 2024-06-26T07:07:08.9746122Z de0f8a8c45c6: Pulling fs layer 2024-06-26T07:07:08.9746537Z 236c8f0245f4: Waiting 2024-06-26T07:07:08.9746902Z 2060028f7488: Pulling fs layer 2024-06-26T07:07:08.9747323Z e797fe86caef: Waiting 2024-06-26T07:07:08.9747707Z e412c7f3b2de: Pulling fs layer 2024-06-26T07:07:08.9748113Z d37dd9ed8948: Waiting 2024-06-26T07:07:08.9748477Z 68af2ec4e30a: Waiting 2024-06-26T07:07:08.9748838Z 967190076fc0: Waiting 2024-06-26T07:07:08.9749242Z 73a0ce681c29: Waiting 2024-06-26T07:07:08.9749587Z 49737bd5ae70: Waiting 2024-06-26T07:07:08.9749946Z 789eff6c51aa: Waiting 2024-06-26T07:07:08.9750302Z 2060028f7488: Waiting 2024-06-26T07:07:08.9750641Z e08e88c13668: Waiting 2024-06-26T07:07:08.9750997Z 12ba7431ff38: Waiting 2024-06-26T07:07:08.9751351Z b599c5a8c7cb: Waiting 2024-06-26T07:07:08.9751713Z 4f4fb700ef54: Waiting 2024-06-26T07:07:08.9752060Z 05a6f6dcc1c0: Waiting 2024-06-26T07:07:08.9752420Z 1b0e27c35f07: Waiting 2024-06-26T07:07:08.9752789Z dda1ce302b9a: Waiting 2024-06-26T07:07:08.9753156Z de0f8a8c45c6: Waiting 2024-06-26T07:07:08.9753501Z cc91933e985a: Waiting 2024-06-26T07:07:08.9753857Z 9f3187916e7d: Waiting 2024-06-26T07:07:08.9754223Z 6bc9dc8315b0: Waiting 2024-06-26T07:07:09.5232223Z 224fe954d725: Verifying Checksum 2024-06-26T07:07:09.5232990Z 224fe954d725: Download complete 2024-06-26T07:07:09.7242258Z 7a2c55901189: Verifying Checksum 2024-06-26T07:07:09.7242863Z 7a2c55901189: Download complete 2024-06-26T07:07:09.7815134Z d527cbbb87e3: Verifying Checksum 2024-06-26T07:07:09.7816118Z d527cbbb87e3: Download complete 2024-06-26T07:07:10.0005467Z b57676e46aee: Download complete 2024-06-26T07:07:10.0343729Z 75722010b82e: Verifying Checksum 2024-06-26T07:07:10.0344695Z 75722010b82e: Download complete 2024-06-26T07:07:10.2849994Z 0c1227890755: Download complete 2024-06-26T07:07:10.3081652Z a41a8d1c11c8: Download complete 2024-06-26T07:07:10.4256977Z 7a2c55901189: Pull complete 2024-06-26T07:07:10.5787322Z d8d1234baab3: Verifying Checksum 2024-06-26T07:07:10.5788338Z d8d1234baab3: Download complete 2024-06-26T07:07:10.6596859Z 224fe954d725: Pull complete 2024-06-26T07:07:10.9285908Z ec1e7978c1fe: Download complete 2024-06-26T07:07:11.2049211Z 309b872da805: Verifying Checksum 2024-06-26T07:07:11.2049846Z 309b872da805: Download complete 2024-06-26T07:07:11.7354936Z 75722010b82e: Pull complete 2024-06-26T07:07:11.7972014Z d527cbbb87e3: Pull complete 2024-06-26T07:07:11.8531682Z b57676e46aee: Pull complete 2024-06-26T07:07:14.3358558Z e82932cb002c: Verifying Checksum 2024-06-26T07:07:14.3359608Z e82932cb002c: Download complete 2024-06-26T07:07:14.5901501Z ca9dd73e3e93: Download complete 2024-06-26T07:07:14.8503070Z 16d76a79d5d6: Download complete 2024-06-26T07:07:15.1143162Z 4bdf41c9ecbe: Verifying Checksum 2024-06-26T07:07:16.5663468Z 4bdf41c9ecbe: Download complete 2024-06-26T07:07:16.5664365Z f4863801e30c: Verifying Checksum 2024-06-26T07:07:16.5664875Z f4863801e30c: Download complete 2024-06-26T07:07:16.8278773Z d37dd9ed8948: Verifying Checksum 2024-06-26T07:07:17.0771917Z d37dd9ed8948: Download complete 2024-06-26T07:07:17.0772492Z cf2f34f225db: Verifying Checksum 2024-06-26T07:07:17.0773259Z cf2f34f225db: Download complete 2024-06-26T07:07:17.3390956Z df546d7b3e34: Verifying Checksum 2024-06-26T07:07:17.3391569Z df546d7b3e34: Download complete 2024-06-26T07:07:29.5176616Z a8c1e85b5e14: Verifying Checksum 2024-06-26T07:07:29.5177599Z a8c1e85b5e14: Download complete 2024-06-26T07:07:29.7819092Z 96d0e2032184: Download complete 2024-06-26T07:07:30.0393388Z 06e12ce59790: Download complete 2024-06-26T07:07:30.3077935Z ed712f63cde7: Verifying Checksum 2024-06-26T07:07:30.3078713Z ed712f63cde7: Download complete 2024-06-26T07:07:30.5724396Z a97a71ef8a4e: Verifying Checksum 2024-06-26T07:07:30.5725406Z a97a71ef8a4e: Download complete 2024-06-26T07:07:30.8275461Z 25cd0c482ad6: Verifying Checksum 2024-06-26T07:07:30.8276286Z 25cd0c482ad6: Download complete 2024-06-26T07:07:32.4267266Z 9bf94bc45658: Verifying Checksum 2024-06-26T07:07:32.4268281Z 9bf94bc45658: Download complete 2024-06-26T07:07:32.6985521Z de45a14c9b22: Download complete 2024-06-26T07:07:32.9624568Z e99241e77209: Verifying Checksum 2024-06-26T07:07:32.9625331Z e99241e77209: Download complete 2024-06-26T07:07:33.2082736Z 35e8f4e2a42f: Verifying Checksum 2024-06-26T07:07:33.2083822Z 35e8f4e2a42f: Download complete 2024-06-26T07:07:33.4833722Z a823ffa4c541: Verifying Checksum 2024-06-26T07:07:33.4834477Z a823ffa4c541: Download complete 2024-06-26T07:07:33.7594749Z 44ba272eae20: Verifying Checksum 2024-06-26T07:07:33.7595768Z 44ba272eae20: Download complete 2024-06-26T07:07:36.0627861Z 7ed32bc8e469: Verifying Checksum 2024-06-26T07:07:36.0628543Z 7ed32bc8e469: Download complete 2024-06-26T07:07:36.3297764Z 73a0ce681c29: Verifying Checksum 2024-06-26T07:07:36.3298680Z 73a0ce681c29: Download complete 2024-06-26T07:07:36.5938862Z 49737bd5ae70: Verifying Checksum 2024-06-26T07:07:36.5939911Z 49737bd5ae70: Download complete 2024-06-26T07:07:37.4918538Z 9cd4c745f8e4: Verifying Checksum 2024-06-26T07:07:37.4919211Z 9cd4c745f8e4: Download complete 2024-06-26T07:07:37.7531011Z b2950a036534: Download complete 2024-06-26T07:07:38.0291271Z 25e19d81d9fc: Verifying Checksum 2024-06-26T07:07:38.0292214Z 25e19d81d9fc: Download complete 2024-06-26T07:07:38.7175871Z 60156b40fcbe: Verifying Checksum 2024-06-26T07:07:38.7176459Z 60156b40fcbe: Download complete 2024-06-26T07:07:38.8773253Z da02d278e524: Verifying Checksum 2024-06-26T07:07:38.8774123Z da02d278e524: Download complete 2024-06-26T07:07:38.9787628Z 7a868239227f: Download complete 2024-06-26T07:07:39.1500067Z 83911d7c7ecf: Verifying Checksum 2024-06-26T07:07:39.1500960Z 83911d7c7ecf: Download complete 2024-06-26T07:07:39.2500836Z b3c5d66c679f: Download complete 2024-06-26T07:07:39.5169447Z c1068d03a811: Verifying Checksum 2024-06-26T07:07:39.5170409Z c1068d03a811: Download complete 2024-06-26T07:07:39.7898189Z 671b4a301d81: Download complete 2024-06-26T07:07:40.7405664Z 67760cf5ec46: Verifying Checksum 2024-06-26T07:07:40.7406703Z 67760cf5ec46: Download complete 2024-06-26T07:07:41.0177996Z ddc9740bcea2: Verifying Checksum 2024-06-26T07:07:41.0178968Z ddc9740bcea2: Download complete 2024-06-26T07:07:41.2921186Z fb2e97768560: Verifying Checksum 2024-06-26T07:07:41.2922539Z fb2e97768560: Download complete 2024-06-26T07:07:41.5529626Z b599c5a8c7cb: Download complete 2024-06-26T07:07:41.8129673Z e08e88c13668: Verifying Checksum 2024-06-26T07:07:41.8130636Z e08e88c13668: Download complete 2024-06-26T07:07:45.5570144Z a8c1e85b5e14: Pull complete 2024-06-26T07:07:46.0855440Z a41a8d1c11c8: Pull complete 2024-06-26T07:07:46.5607413Z 0c1227890755: Pull complete 2024-06-26T07:07:46.9640264Z d8d1234baab3: Pull complete 2024-06-26T07:08:02.2081736Z 3d914950f8ad: Verifying Checksum 2024-06-26T07:08:02.2082640Z 3d914950f8ad: Download complete 2024-06-26T07:08:02.4745206Z cf7c2cb896cb: Download complete 2024-06-26T07:08:02.7505631Z 3a778ddfac2a: Verifying Checksum 2024-06-26T07:08:02.7506463Z 3a778ddfac2a: Download complete 2024-06-26T07:08:03.0269706Z 236c8f0245f4: Verifying Checksum 2024-06-26T07:08:03.0270717Z 236c8f0245f4: Download complete 2024-06-26T07:08:03.1419439Z 4f4fb700ef54: Verifying Checksum 2024-06-26T07:08:03.1420390Z 4f4fb700ef54: Download complete 2024-06-26T07:08:03.4133207Z dda1ce302b9a: Download complete 2024-06-26T07:08:03.6677297Z e797fe86caef: Download complete 2024-06-26T07:08:04.1791929Z 967190076fc0: Download complete 2024-06-26T07:08:04.5071555Z 4fb0a83f0685: Download complete 2024-06-26T07:08:04.8185120Z 68af2ec4e30a: Verifying Checksum 2024-06-26T07:08:04.8185941Z 68af2ec4e30a: Download complete 2024-06-26T07:08:05.1050846Z cc91933e985a: Verifying Checksum 2024-06-26T07:08:05.1051828Z cc91933e985a: Download complete 2024-06-26T07:08:05.3757553Z 9f3187916e7d: Verifying Checksum 2024-06-26T07:08:05.3758191Z 9f3187916e7d: Download complete 2024-06-26T07:08:05.9018862Z 526083994857: Verifying Checksum 2024-06-26T07:08:05.9019903Z 526083994857: Download complete 2024-06-26T07:08:06.1789490Z 05a6f6dcc1c0: Verifying Checksum 2024-06-26T07:08:06.1790445Z 05a6f6dcc1c0: Download complete 2024-06-26T07:08:07.2143081Z 6bc9dc8315b0: Verifying Checksum 2024-06-26T07:08:07.2143670Z 6bc9dc8315b0: Download complete 2024-06-26T07:08:07.4895042Z 1b0e27c35f07: Download complete 2024-06-26T07:08:07.6530816Z 12ba7431ff38: Verifying Checksum 2024-06-26T07:08:07.6531800Z 12ba7431ff38: Download complete 2024-06-26T07:08:07.9237549Z 789eff6c51aa: Verifying Checksum 2024-06-26T07:08:07.9238390Z 789eff6c51aa: Download complete 2024-06-26T07:08:08.1768888Z de0f8a8c45c6: Verifying Checksum 2024-06-26T07:08:08.1769885Z de0f8a8c45c6: Download complete 2024-06-26T07:08:09.1787413Z 2060028f7488: Verifying Checksum 2024-06-26T07:08:09.1788400Z 2060028f7488: Download complete 2024-06-26T07:08:09.4471773Z e412c7f3b2de: Verifying Checksum 2024-06-26T07:08:21.0529874Z 7ed32bc8e469: Pull complete 2024-06-26T07:08:21.1202076Z ec1e7978c1fe: Pull complete 2024-06-26T07:08:21.1791331Z 309b872da805: Pull complete 2024-06-26T07:08:21.6772821Z 95237e7311a1: Verifying Checksum 2024-06-26T07:08:21.6775371Z 95237e7311a1: Download complete 2024-06-26T07:08:27.0028710Z e82932cb002c: Pull complete 2024-06-26T07:08:27.0580242Z ca9dd73e3e93: Pull complete 2024-06-26T07:08:27.1113398Z 16d76a79d5d6: Pull complete 2024-06-26T07:08:27.1681678Z 4bdf41c9ecbe: Pull complete 2024-06-26T07:08:29.2233966Z f4863801e30c: Pull complete 2024-06-26T07:08:29.2762953Z d37dd9ed8948: Pull complete 2024-06-26T07:08:29.3321227Z cf2f34f225db: Pull complete 2024-06-26T07:08:29.3819422Z df546d7b3e34: Pull complete 2024-06-26T07:08:32.0783788Z 84b2c7a89982: Verifying Checksum 2024-06-26T07:08:32.0784873Z 84b2c7a89982: Download complete 2024-06-26T07:09:13.3001792Z 3d914950f8ad: Pull complete 2024-06-26T07:09:13.7689997Z 96d0e2032184: Pull complete 2024-06-26T07:09:14.2054619Z 06e12ce59790: Pull complete 2024-06-26T07:09:14.6672158Z ed712f63cde7: Pull complete 2024-06-26T07:09:15.1415468Z a97a71ef8a4e: Pull complete 2024-06-26T07:09:15.3741808Z 25cd0c482ad6: Pull complete 2024-06-26T07:09:18.8790277Z 9bf94bc45658: Pull complete 2024-06-26T07:09:18.9287726Z de45a14c9b22: Pull complete 2024-06-26T07:09:18.9782150Z e99241e77209: Pull complete 2024-06-26T07:09:19.0311804Z 35e8f4e2a42f: Pull complete 2024-06-26T07:09:19.0830073Z a823ffa4c541: Pull complete 2024-06-26T07:09:19.1340233Z 44ba272eae20: Pull complete 2024-06-26T07:09:25.8880184Z da02d278e524: Pull complete 2024-06-26T07:09:25.9398179Z 73a0ce681c29: Pull complete 2024-06-26T07:09:25.9912828Z 49737bd5ae70: Pull complete 2024-06-26T07:09:26.8498611Z 9cd4c745f8e4: Pull complete 2024-06-26T07:09:26.9007386Z b2950a036534: Pull complete 2024-06-26T07:09:26.9526519Z 25e19d81d9fc: Pull complete 2024-06-26T07:09:27.2874126Z 60156b40fcbe: Pull complete 2024-06-26T07:09:27.3448576Z 7a868239227f: Pull complete 2024-06-26T07:09:27.4494295Z 83911d7c7ecf: Pull complete 2024-06-26T07:09:27.5015637Z b3c5d66c679f: Pull complete 2024-06-26T07:10:09.8682274Z 84b2c7a89982: Pull complete 2024-06-26T07:10:09.9289347Z c1068d03a811: Pull complete 2024-06-26T07:10:09.9803191Z 671b4a301d81: Pull complete 2024-06-26T07:10:11.5338085Z 67760cf5ec46: Pull complete 2024-06-26T07:10:11.9440120Z ddc9740bcea2: Pull complete 2024-06-26T07:10:12.4233611Z fb2e97768560: Pull complete 2024-06-26T07:10:12.9733662Z b599c5a8c7cb: Pull complete 2024-06-26T07:10:13.0770593Z e08e88c13668: Pull complete 2024-06-26T07:10:48.8458912Z 12ba7431ff38: Pull complete 2024-06-26T07:10:49.2415569Z cf7c2cb896cb: Pull complete 2024-06-26T07:10:49.7314361Z 3a778ddfac2a: Pull complete 2024-06-26T07:10:50.6577663Z 236c8f0245f4: Pull complete 2024-06-26T07:10:50.7082073Z 4f4fb700ef54: Pull complete 2024-06-26T07:10:50.7579954Z dda1ce302b9a: Pull complete 2024-06-26T07:10:50.8070261Z e797fe86caef: Pull complete 2024-06-26T07:10:51.0239081Z 967190076fc0: Pull complete 2024-06-26T07:10:51.0753409Z 4fb0a83f0685: Pull complete 2024-06-26T07:10:51.1248621Z 68af2ec4e30a: Pull complete 2024-06-26T07:10:51.1743192Z cc91933e985a: Pull complete 2024-06-26T07:10:51.2231809Z 9f3187916e7d: Pull complete 2024-06-26T07:10:51.6839685Z 526083994857: Pull complete 2024-06-26T07:10:51.7335433Z 05a6f6dcc1c0: Pull complete 2024-06-26T07:10:53.4905430Z 6bc9dc8315b0: Pull complete 2024-06-26T07:10:53.5399097Z 1b0e27c35f07: Pull complete 2024-06-26T07:11:09.7240254Z 95237e7311a1: Pull complete 2024-06-26T07:11:09.7767260Z 789eff6c51aa: Pull complete 2024-06-26T07:11:09.8291441Z de0f8a8c45c6: Pull complete 2024-06-26T07:11:10.6220762Z 2060028f7488: Pull complete 2024-06-26T07:11:10.6721997Z e412c7f3b2de: Pull complete 2024-06-26T07:11:10.8852421Z Digest: sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:11:10.8882394Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:11:10.8916488Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:11:10.9025198Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:11:10.9026599Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:11:10.9042146Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:10.9042768Z env: 2024-06-26T07:11:10.9043074Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:10.9043482Z ##[endgroup] 2024-06-26T07:11:10.9283864Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-06-26T07:11:10.9284538Z with: 2024-06-26T07:11:10.9284853Z driver-version: 550.54.15 2024-06-26T07:11:10.9285263Z env: 2024-06-26T07:11:10.9285577Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:10.9285971Z ##[endgroup] 2024-06-26T07:11:10.9340035Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:11:10.9340717Z with: 2024-06-26T07:11:10.9341028Z timeout_minutes: 10 2024-06-26T07:11:10.9341406Z max_attempts: 3 2024-06-26T07:11:10.9384901Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2024-06-26T07:11:10.9428554Z retry_wait_seconds: 10 2024-06-26T07:11:10.9428990Z polling_interval_seconds: 1 2024-06-26T07:11:10.9429481Z warning_on_retry: true 2024-06-26T07:11:10.9429878Z continue_on_error: false 2024-06-26T07:11:10.9430273Z env: 2024-06-26T07:11:10.9430582Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:10.9431005Z DRIVER_VERSION: 550.54.15 2024-06-26T07:11:10.9431399Z ##[endgroup] 2024-06-26T07:11:11.0165131Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-06-26T07:11:11.0166389Z + install_nvidia_driver_common 2024-06-26T07:11:11.0169525Z + echo 'Before installing NVIDIA driver' 2024-06-26T07:11:11.0171422Z Before installing NVIDIA driver 2024-06-26T07:11:11.0172076Z + lspci 2024-06-26T07:11:11.0284769Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-06-26T07:11:11.0285988Z 00:01.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 03) 2024-06-26T07:11:11.0286924Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 2024-06-26T07:11:11.0288344Z 00:03.0 Non-VGA unclassified device: Red Hat, Inc. Virtio SCSI 2024-06-26T07:11:11.0289219Z 00:04.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) 2024-06-26T07:11:11.0290044Z 00:05.0 Ethernet controller: Red Hat, Inc. Virtio network device 2024-06-26T07:11:11.0290874Z 00:06.0 Unclassified device [00ff]: Red Hat, Inc. Virtio RNG 2024-06-26T07:11:11.0291480Z + lsmod 2024-06-26T07:11:11.0310079Z Module Size Used by 2024-06-26T07:11:11.0310819Z nvidia_modeset 1343488 0 2024-06-26T07:11:11.0311278Z nvidia_uvm 4673536 0 2024-06-26T07:11:11.0311820Z nvidia 54018048 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:11:11.0314353Z drm 618496 1 nvidia 2024-06-26T07:11:11.0314981Z ib_core 393216 0 2024-06-26T07:11:11.0315450Z btrfs 1540096 0 2024-06-26T07:11:11.0315899Z blake2b_generic 20480 0 2024-06-26T07:11:11.0316381Z xor 24576 1 btrfs 2024-06-26T07:11:11.0316881Z zstd_compress 225280 1 btrfs 2024-06-26T07:11:11.0317402Z raid6_pq 122880 1 btrfs 2024-06-26T07:11:11.0317974Z ufs 106496 0 2024-06-26T07:11:11.0318420Z msdos 20480 0 2024-06-26T07:11:11.0318869Z xfs 1753088 0 2024-06-26T07:11:11.0319583Z veth 32768 0 2024-06-26T07:11:11.0320035Z xt_conntrack 16384 1 2024-06-26T07:11:11.0320522Z xt_MASQUERADE 20480 1 2024-06-26T07:11:11.0320975Z xfrm_user 40960 1 2024-06-26T07:11:11.0321424Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:11:11.0322243Z xt_addrtype 16384 2 2024-06-26T07:11:11.0322970Z iptable_filter 16384 1 2024-06-26T07:11:11.0323421Z iptable_nat 16384 1 2024-06-26T07:11:11.0324014Z nf_nat 49152 2 iptable_nat,xt_MASQUERADE 2024-06-26T07:11:11.0324618Z bpfilter 16384 0 2024-06-26T07:11:11.0325312Z br_netfilter 28672 0 2024-06-26T07:11:11.0325801Z bridge 307200 1 br_netfilter 2024-06-26T07:11:11.0326328Z stp 16384 1 bridge 2024-06-26T07:11:11.0326985Z llc 16384 2 bridge,stp 2024-06-26T07:11:11.0327951Z aufs 270336 0 2024-06-26T07:11:11.0328787Z overlay 151552 0 2024-06-26T07:11:11.0329469Z nls_iso8859_1 16384 1 2024-06-26T07:11:11.0330139Z dm_multipath 40960 0 2024-06-26T07:11:11.0330617Z scsi_dh_rdac 16384 0 2024-06-26T07:11:11.0331066Z scsi_dh_emc 16384 0 2024-06-26T07:11:11.0331508Z scsi_dh_alua 20480 0 2024-06-26T07:11:11.0331949Z binfmt_misc 24576 1 2024-06-26T07:11:11.0332404Z crct10dif_pclmul 16384 1 2024-06-26T07:11:11.0332845Z crc32_pclmul 16384 0 2024-06-26T07:11:11.0333288Z ghash_clmulni_intel 16384 0 2024-06-26T07:11:11.0333749Z aesni_intel 376832 0 2024-06-26T07:11:11.0334315Z virtio_net 61440 0 2024-06-26T07:11:11.0334776Z net_failover 20480 1 virtio_net 2024-06-26T07:11:11.0335588Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:11:11.0336541Z cryptd 24576 2 crypto_simd,ghash_clmulni_intel 2024-06-26T07:11:11.0337698Z failover 16384 1 net_failover 2024-06-26T07:11:11.0338577Z input_leds 16384 0 2024-06-26T07:11:11.0339386Z psmouse 176128 0 2024-06-26T07:11:11.0340263Z serio_raw 20480 0 2024-06-26T07:11:11.0340700Z sch_fq_codel 24576 13 2024-06-26T07:11:11.0341157Z efi_pstore 16384 0 2024-06-26T07:11:11.0341593Z virtio_rng 16384 0 2024-06-26T07:11:11.0342150Z ip_tables 32768 2 iptable_filter,iptable_nat 2024-06-26T07:11:11.0343112Z x_tables 53248 6 xt_conntrack,iptable_filter,xt_addrtype,ip_tables,iptable_nat,xt_MASQUERADE 2024-06-26T07:11:11.0344350Z autofs4 49152 2 2024-06-26T07:11:11.0344802Z + modinfo nvidia 2024-06-26T07:11:11.0345591Z filename: /lib/modules/5.15.0-1047-gcp/kernel/drivers/video/nvidia.ko 2024-06-26T07:11:11.0346353Z alias: char-major-195-* 2024-06-26T07:11:11.0346832Z version: 550.54.15 2024-06-26T07:11:11.0347241Z supported: external 2024-06-26T07:11:11.0347642Z license: NVIDIA 2024-06-26T07:11:11.0348082Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:11:11.0348665Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:11:11.0349231Z srcversion: 1BB26F5DEAC3FABF74093C6 2024-06-26T07:11:11.0349792Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:11:11.0350447Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:11:11.0351050Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:11:11.0351576Z depends: drm 2024-06-26T07:11:11.0351943Z retpoline: Y 2024-06-26T07:11:11.0352286Z name: nvidia 2024-06-26T07:11:11.0352889Z vermagic: 5.15.0-1047-gcp SMP mod_unload modversions 2024-06-26T07:11:11.0353634Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:11:11.0354493Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:11:11.0355218Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:11:11.0355753Z parm: NVreg_RmLogonRC:int 2024-06-26T07:11:11.0356520Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:11:11.0357073Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:11:11.0357597Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:11:11.0358125Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:11:11.0358774Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:11:11.0359461Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:11:11.0360044Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:11:11.0360560Z parm: NVreg_EnableMSI:int 2024-06-26T07:11:11.0361057Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:11:11.0361782Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:11:11.0362432Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:11:11.0363153Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:11:11.0363891Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:11:11.0364642Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:11.0365381Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:11:11.0366147Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:11.0366887Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:11:11.0367471Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:11:11.0368124Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:11:11.0368802Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:11:11.0369406Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:11:11.0369953Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:11:11.0370549Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:11:11.0371119Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:11:11.0371675Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:11:11.0372295Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:11:11.0372934Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:11:11.0373515Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:11:11.0374126Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:11:11.0374730Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:11:11.0375321Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:11:11.0375926Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:11:11.0376492Z parm: NVreg_RmMsg:charp 2024-06-26T07:11:11.0376983Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:11:11.0377547Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:11:11.0378116Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:11:11.0378659Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:11:11.0379237Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:11:11.0379818Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:11:11.0380368Z parm: rm_firmware_active:charp 2024-06-26T07:11:11.0380852Z + HAS_NVIDIA_DRIVER=0 2024-06-26T07:11:11.0381336Z ++ command -v nvidia-smi 2024-06-26T07:11:11.0381817Z + '[' -x /usr/bin/nvidia-smi ']' 2024-06-26T07:11:11.0382238Z + set +e 2024-06-26T07:11:11.0382864Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-06-26T07:11:11.0673186Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-06-26T07:11:11.0674094Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:11:11.0675101Z + '[' 0 -ne 0 ']' 2024-06-26T07:11:11.0675920Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-06-26T07:11:11.0676775Z + HAS_NVIDIA_DRIVER=1 2024-06-26T07:11:11.0677754Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-06-26T07:11:11.0678655Z + set -e 2024-06-26T07:11:11.0679069Z + '[' 1 -eq 0 ']' 2024-06-26T07:11:11.0679761Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-06-26T07:11:11.0680740Z + post_install_nvidia_driver_common 2024-06-26T07:11:11.0681254Z + sudo modprobe nvidia 2024-06-26T07:11:11.0804169Z + echo 'After installing NVIDIA driver' 2024-06-26T07:11:11.0805054Z + lspci 2024-06-26T07:11:11.0805408Z After installing NVIDIA driver 2024-06-26T07:11:11.0915970Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-06-26T07:11:11.0917336Z 00:01.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 03) 2024-06-26T07:11:11.0918226Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 2024-06-26T07:11:11.0919166Z 00:03.0 Non-VGA unclassified device: Red Hat, Inc. Virtio SCSI 2024-06-26T07:11:11.0919970Z 00:04.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) 2024-06-26T07:11:11.0920775Z 00:05.0 Ethernet controller: Red Hat, Inc. Virtio network device 2024-06-26T07:11:11.0921878Z 00:06.0 Unclassified device [00ff]: Red Hat, Inc. Virtio RNG 2024-06-26T07:11:11.0922477Z + lsmod 2024-06-26T07:11:11.0941120Z Module Size Used by 2024-06-26T07:11:11.0942064Z nvidia_modeset 1343488 0 2024-06-26T07:11:11.0942925Z nvidia_uvm 4673536 0 2024-06-26T07:11:11.0943920Z nvidia 54018048 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:11:11.0945069Z drm 618496 1 nvidia 2024-06-26T07:11:11.0945544Z ib_core 393216 0 2024-06-26T07:11:11.0945961Z btrfs 1540096 0 2024-06-26T07:11:11.0946397Z blake2b_generic 20480 0 2024-06-26T07:11:11.0946842Z xor 24576 1 btrfs 2024-06-26T07:11:11.0947318Z zstd_compress 225280 1 btrfs 2024-06-26T07:11:11.0947781Z raid6_pq 122880 1 btrfs 2024-06-26T07:11:11.0948230Z ufs 106496 0 2024-06-26T07:11:11.0948656Z msdos 20480 0 2024-06-26T07:11:11.0949150Z xfs 1753088 0 2024-06-26T07:11:11.0949594Z veth 32768 0 2024-06-26T07:11:11.0950024Z xt_conntrack 16384 1 2024-06-26T07:11:11.0950461Z xt_MASQUERADE 20480 1 2024-06-26T07:11:11.0950892Z xfrm_user 40960 1 2024-06-26T07:11:11.0951328Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:11:11.0951831Z xt_addrtype 16384 2 2024-06-26T07:11:11.0952273Z iptable_filter 16384 1 2024-06-26T07:11:11.0952715Z iptable_nat 16384 1 2024-06-26T07:11:11.0953232Z nf_nat 49152 2 iptable_nat,xt_MASQUERADE 2024-06-26T07:11:11.0953806Z bpfilter 16384 0 2024-06-26T07:11:11.0954247Z br_netfilter 28672 0 2024-06-26T07:11:11.0954727Z bridge 307200 1 br_netfilter 2024-06-26T07:11:11.0955239Z stp 16384 1 bridge 2024-06-26T07:11:11.0955730Z llc 16384 2 bridge,stp 2024-06-26T07:11:11.0956225Z aufs 270336 0 2024-06-26T07:11:11.0956648Z overlay 151552 0 2024-06-26T07:11:11.0957086Z nls_iso8859_1 16384 1 2024-06-26T07:11:11.0957520Z dm_multipath 40960 0 2024-06-26T07:11:11.0957956Z scsi_dh_rdac 16384 0 2024-06-26T07:11:11.0958372Z scsi_dh_emc 16384 0 2024-06-26T07:11:11.0958803Z scsi_dh_alua 20480 0 2024-06-26T07:11:11.0959301Z binfmt_misc 24576 1 2024-06-26T07:11:11.0959743Z crct10dif_pclmul 16384 1 2024-06-26T07:11:11.0960169Z crc32_pclmul 16384 0 2024-06-26T07:11:11.0960608Z ghash_clmulni_intel 16384 0 2024-06-26T07:11:11.0961090Z aesni_intel 376832 0 2024-06-26T07:11:11.0961878Z virtio_net 61440 0 2024-06-26T07:11:11.0962663Z net_failover 20480 1 virtio_net 2024-06-26T07:11:11.0963688Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:11:11.0964361Z cryptd 24576 2 crypto_simd,ghash_clmulni_intel 2024-06-26T07:11:11.0964993Z failover 16384 1 net_failover 2024-06-26T07:11:11.0965521Z input_leds 16384 0 2024-06-26T07:11:11.0965961Z psmouse 176128 0 2024-06-26T07:11:11.0966397Z serio_raw 20480 0 2024-06-26T07:11:11.0966818Z sch_fq_codel 24576 13 2024-06-26T07:11:11.0967258Z efi_pstore 16384 0 2024-06-26T07:11:11.0967988Z virtio_rng 16384 0 2024-06-26T07:11:11.0968553Z ip_tables 32768 2 iptable_filter,iptable_nat 2024-06-26T07:11:11.0969566Z x_tables 53248 6 xt_conntrack,iptable_filter,xt_addrtype,ip_tables,iptable_nat,xt_MASQUERADE 2024-06-26T07:11:11.0970423Z autofs4 49152 2 2024-06-26T07:11:11.0970862Z + modinfo nvidia 2024-06-26T07:11:11.0971723Z filename: /lib/modules/5.15.0-1047-gcp/kernel/drivers/video/nvidia.ko 2024-06-26T07:11:11.0972486Z alias: char-major-195-* 2024-06-26T07:11:11.0972939Z version: 550.54.15 2024-06-26T07:11:11.0973341Z supported: external 2024-06-26T07:11:11.0973898Z license: NVIDIA 2024-06-26T07:11:11.0974321Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:11:11.0974908Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:11:11.0975473Z srcversion: 1BB26F5DEAC3FABF74093C6 2024-06-26T07:11:11.0976040Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:11:11.0976639Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:11:11.0977232Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:11:11.0977748Z depends: drm 2024-06-26T07:11:11.0978095Z retpoline: Y 2024-06-26T07:11:11.0978447Z name: nvidia 2024-06-26T07:11:11.0979101Z vermagic: 5.15.0-1047-gcp SMP mod_unload modversions 2024-06-26T07:11:11.0979830Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:11:11.0980611Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:11:11.0981347Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:11:11.0981879Z parm: NVreg_RmLogonRC:int 2024-06-26T07:11:11.0982402Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:11:11.0993668Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:11:11.0994191Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:11:11.0994712Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:11:11.0995369Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:11:11.0996048Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:11:11.0996626Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:11:11.0997138Z parm: NVreg_EnableMSI:int 2024-06-26T07:11:11.0997632Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:11:11.0998163Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:11:11.0998809Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:11:11.0999584Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:11:11.1000256Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:11:11.1001016Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:11.1001755Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:11:11.1002512Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:11:11.1003235Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:11:11.1003835Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:11:11.1004483Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:11:11.1005155Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:11:11.1005740Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:11:11.1006292Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:11:11.1006871Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:11:11.1007437Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:11:11.1007958Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:11:11.1008566Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:11:11.1009253Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:11:11.1009827Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:11:11.1010397Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:11:11.1010999Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:11:11.1011587Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:11:11.1012412Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:11:11.1012988Z parm: NVreg_RmMsg:charp 2024-06-26T07:11:11.1013471Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:11:11.1014027Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:11:11.1014575Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:11:11.1015116Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:11:11.1015682Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:11:11.1016258Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:11:11.1016781Z parm: rm_firmware_active:charp 2024-06-26T07:11:11.1017382Z + set +e 2024-06-26T07:11:11.1017779Z + nvidia-smi 2024-06-26T07:11:11.1194515Z Wed Jun 26 07:11:11 2024 2024-06-26T07:11:11.1195596Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:11.1196644Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-06-26T07:11:11.1197660Z |-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:11:11.1198681Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-06-26T07:11:11.1199883Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-06-26T07:11:11.1200749Z | | | MIG M. | 2024-06-26T07:11:11.1201420Z |=========================================+========================+======================| 2024-06-26T07:11:11.1336709Z | 0 NVIDIA A100-SXM4-40GB On | 00000000:00:04.0 Off | 0 | 2024-06-26T07:11:11.1337680Z | N/A 31C P0 46W / 400W | 0MiB / 40960MiB | 0% Default | 2024-06-26T07:11:11.1338692Z | | | Disabled | 2024-06-26T07:11:11.1339624Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:11:11.1340354Z 2024-06-26T07:11:11.1341187Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:11.1341961Z | Processes: | 2024-06-26T07:11:11.1342839Z | GPU GI CI PID Type Process name GPU Memory | 2024-06-26T07:11:11.1343702Z | ID ID Usage | 2024-06-26T07:11:11.1344573Z |=========================================================================================| 2024-06-26T07:11:11.1345389Z | No running processes found | 2024-06-26T07:11:11.1346352Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:11.5234716Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-06-26T07:11:11.5429894Z NVIDIA A100-SXM4-40GB 2024-06-26T07:11:11.5507317Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:11:11.5507884Z + '[' 0 -eq 0 ']' 2024-06-26T07:11:11.5508367Z + echo 'INFO: Ignoring allowed status 0' 2024-06-26T07:11:11.5508864Z + set -e 2024-06-26T07:11:11.5509205Z INFO: Ignoring allowed status 0 2024-06-26T07:11:11.5517381Z == Installing nvidia container toolkit for ubuntu20.04 == 2024-06-26T07:11:11.5525115Z ++ dpkg-query -W '--showformat=${db:Status-Status}' nvidia-docker2 2024-06-26T07:11:11.5632839Z + status=installed 2024-06-26T07:11:11.5633381Z + '[' '!' 0 = 0 ']' 2024-06-26T07:11:11.5633835Z + '[' '!' installed = installed ']' 2024-06-26T07:11:12.0153337Z Command completed after 1 attempt(s). 2024-06-26T07:11:12.0451689Z ##[group]Run sudo nvidia-smi -pm 1 2024-06-26T07:11:12.0452666Z sudo nvidia-smi -pm 1 2024-06-26T07:11:12.0453536Z sudo nvidia-smi -ac 1215,1410 2024-06-26T07:11:12.0454393Z nvidia-smi 2024-06-26T07:11:12.0473963Z shell: /usr/bin/bash -e {0} 2024-06-26T07:11:12.0474739Z env: 2024-06-26T07:11:12.0475281Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:12.0476214Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:12.0477267Z ##[endgroup] 2024-06-26T07:11:12.1354038Z Persistence mode is already Enabled for GPU 00000000:00:04.0. 2024-06-26T07:11:12.1354742Z All done. 2024-06-26T07:11:12.1726277Z Applications clocks set to "(MEM 1215, SM 1410)" for GPU 00000000:00:04.0 2024-06-26T07:11:12.1727034Z All done. 2024-06-26T07:11:12.1952701Z Wed Jun 26 07:11:12 2024 2024-06-26T07:11:12.1954641Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:12.1956047Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-06-26T07:11:12.1957219Z |-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:11:12.1958293Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-06-26T07:11:12.1959814Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-06-26T07:11:12.1960751Z | | | MIG M. | 2024-06-26T07:11:12.1961431Z |=========================================+========================+======================| 2024-06-26T07:11:12.2025421Z | 0 NVIDIA A100-SXM4-40GB On | 00000000:00:04.0 Off | 0 | 2024-06-26T07:11:12.2027149Z | N/A 32C P0 67W / 400W | 0MiB / 40960MiB | 0% Default | 2024-06-26T07:11:12.2028533Z | | | Disabled | 2024-06-26T07:11:12.2029531Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:11:12.2030265Z 2024-06-26T07:11:12.2031094Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:12.2031890Z | Processes: | 2024-06-26T07:11:12.2032777Z | GPU GI CI PID Type Process name GPU Memory | 2024-06-26T07:11:12.2033639Z | ID ID Usage | 2024-06-26T07:11:12.2034334Z |=========================================================================================| 2024-06-26T07:11:12.2035171Z | No running processes found | 2024-06-26T07:11:12.2036148Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:11:12.4658324Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:11:12.4660098Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:11:12.4661621Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-06-26T07:11:12.4663146Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:11:12.4683946Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:12.4684927Z env: 2024-06-26T07:11:12.4685394Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:12.4686186Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:12.4687040Z ##[endgroup] 2024-06-26T07:11:13.7047822Z Requirement already satisfied: psutil==5.9.1 in /home/ubuntu/.local/lib/python3.8/site-packages (5.9.1) 2024-06-26T07:11:13.7136479Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ubuntu/.local/lib/python3.8/site-packages (11.525.84) 2024-06-26T07:11:13.9326728Z Prepare all required actions 2024-06-26T07:11:13.9327773Z Getting action download info 2024-06-26T07:11:14.0711193Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-06-26T07:11:14.5578567Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-06-26T07:11:14.8910218Z ##[group]Run ./.github/actions/download-build-artifacts 2024-06-26T07:11:14.8910803Z with: 2024-06-26T07:11:14.8911203Z name: linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-06-26T07:11:14.8911742Z s3-bucket: gha-artifacts 2024-06-26T07:11:14.8912135Z env: 2024-06-26T07:11:14.8912443Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:14.8912971Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:14.8913745Z ##[endgroup] 2024-06-26T07:11:14.8955253Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:11:14.8955798Z with: 2024-06-26T07:11:14.8956249Z name: linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-06-26T07:11:14.8956786Z s3-bucket: gha-artifacts 2024-06-26T07:11:14.8957200Z region: us-east-1 2024-06-26T07:11:14.8957544Z env: 2024-06-26T07:11:14.8957854Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:14.8958394Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:14.8958977Z ##[endgroup] 2024-06-26T07:11:15.4789752Z (node:3596936) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:11:15.4790686Z 2024-06-26T07:11:15.4791007Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:11:15.4791912Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:11:15.4793578Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:11:15.6099451Z Found 1 objects with prefix pytorch/pytorch/9673646806/linux-focal-cuda12.1-py3.10-gcc9-sm80/ 2024-06-26T07:11:15.6100930Z Starting download (1/1): /home/weiwangmeta/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:11:43.2404810Z Finished download (1/1): /home/weiwangmeta/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:11:43.2407986Z Artifact download has finished successfully 2024-06-26T07:11:43.2691956Z ##[group]Run unzip -o artifacts.zip 2024-06-26T07:11:43.2692493Z unzip -o artifacts.zip 2024-06-26T07:11:43.2706775Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:43.2707408Z env: 2024-06-26T07:11:43.2707734Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:43.2708282Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:43.2708890Z ##[endgroup] 2024-06-26T07:11:43.2772292Z Archive: artifacts.zip 2024-06-26T07:11:43.2774804Z creating: dist/ 2024-06-26T07:11:45.6564049Z inflating: dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:11:45.6565380Z creating: build/custom_test_artifacts/ 2024-06-26T07:11:45.6566720Z creating: build/custom_test_artifacts/custom-op-build/ 2024-06-26T07:11:45.6567635Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-06-26T07:11:45.6568715Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:11:45.6572126Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:11:45.6573846Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-06-26T07:11:45.6575298Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:11:45.6576854Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:11:45.6578156Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:11:45.6579608Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:11:45.6581484Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:11:45.6582800Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:11:45.6584288Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:11:45.6585869Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:11:45.6587389Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:11:45.6588884Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:11:45.6590370Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:11:45.6592100Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:11:45.6593634Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:11:45.6595006Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:11:45.6596341Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:11:45.6635259Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:11:45.6680880Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:11:45.6682779Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:11:45.6735218Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:11:45.6737155Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:11:45.6739044Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:11:45.6740992Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:11:45.6742934Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:11:45.6744953Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:11:45.6746854Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:11:45.6748701Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:11:45.6750485Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:11:45.6752176Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:11:45.6753797Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:11:45.6755387Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:11:45.6757036Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:11:45.6758574Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:11:45.6760165Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:11:45.6816909Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:11:45.6888544Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:11:45.6890087Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:11:45.6891383Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:11:45.6892503Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:11:45.6893677Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-06-26T07:11:45.6894889Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-06-26T07:11:45.6896321Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-06-26T07:11:45.6898065Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-06-26T07:11:45.6899510Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-06-26T07:11:45.6900853Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-06-26T07:11:45.6902333Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-06-26T07:11:45.6903734Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-06-26T07:11:45.6905326Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-06-26T07:11:45.6906787Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-06-26T07:11:45.6908158Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-06-26T07:11:45.6919796Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-06-26T07:11:45.7078355Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-06-26T07:11:45.7079652Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-06-26T07:11:45.7081082Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-06-26T07:11:45.7082650Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-06-26T07:11:45.7084170Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-06-26T07:11:45.7085604Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-06-26T07:11:45.7087133Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-06-26T07:11:45.7088643Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-06-26T07:11:45.7090118Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-06-26T07:11:45.7091604Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-06-26T07:11:45.7093061Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-06-26T07:11:45.7108180Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-06-26T07:11:45.7206459Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-06-26T07:11:45.7208012Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:11:45.7209438Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:11:45.7210973Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-06-26T07:11:45.7212181Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-06-26T07:11:45.7213338Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-06-26T07:11:45.7214490Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-06-26T07:11:45.7215586Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-06-26T07:11:45.7216649Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-06-26T07:11:45.7217692Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-06-26T07:11:45.7348924Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-06-26T07:11:45.7422998Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-06-26T07:11:45.7424084Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-06-26T07:11:45.7425001Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-06-26T07:11:45.7426111Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:11:45.7432079Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:11:45.7433279Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-06-26T07:11:45.7434469Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:11:45.7435786Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:11:45.7437055Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:11:45.7438549Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:11:45.7439985Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:11:45.7441258Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:11:45.7442519Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:11:45.7443996Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:11:45.7445491Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:11:45.7447025Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:11:45.7448505Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:11:45.7449998Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:11:45.7451497Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:11:45.7452828Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:11:45.7454109Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:11:45.7495253Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:11:45.7541509Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:11:45.7543435Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:11:45.7595958Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:11:45.7597827Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:11:45.7600001Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:11:45.7601939Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:11:45.7603791Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:11:45.7605689Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:11:45.7607506Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:11:45.7609318Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:11:45.7611298Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:11:45.7613087Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:11:45.7614710Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:11:45.7616333Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:11:45.7617941Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:11:45.7619494Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:11:45.7621070Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:11:45.7676942Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:11:45.7748744Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:11:45.7750313Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:11:45.7751590Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:11:45.7752697Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:11:45.7753847Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-06-26T07:11:45.7755054Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-06-26T07:11:45.7756503Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-06-26T07:11:45.7758063Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-06-26T07:11:45.7759565Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-06-26T07:11:45.7760957Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-06-26T07:11:45.7762380Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-06-26T07:11:45.7763817Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-06-26T07:11:45.7765259Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-06-26T07:11:45.7766757Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-06-26T07:11:45.7768181Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-06-26T07:11:45.7779779Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-06-26T07:11:45.7855801Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-06-26T07:11:45.7857367Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:11:45.7858763Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:11:45.7859996Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-06-26T07:11:45.7861148Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-06-26T07:11:45.7862303Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-06-26T07:11:45.7863453Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-06-26T07:11:45.7864858Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-06-26T07:11:45.7865873Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-06-26T07:11:45.7866899Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-06-26T07:11:45.7921685Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-06-26T07:11:45.7922644Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-06-26T07:11:45.7923598Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-06-26T07:11:45.7924734Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:11:45.7931256Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:11:45.7932538Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-06-26T07:11:45.7933846Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:11:45.7935209Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:11:45.7936612Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:11:45.7938132Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:11:45.7939664Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:11:45.7941030Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:11:45.7942404Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:11:45.7944118Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:11:45.7945770Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:11:45.7947354Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:11:45.7948919Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:11:45.7950532Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:11:45.7952135Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:11:45.7953564Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:11:45.7954949Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:11:45.7993912Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:11:45.8040731Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:11:45.8043043Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:11:45.8094271Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:11:45.8096324Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:11:45.8098335Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:11:45.8100376Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:11:45.8102458Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:11:45.8104757Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:11:45.8106772Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:11:45.8108685Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:11:45.8110532Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:11:45.8112308Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:11:45.8114003Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:11:45.8115733Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:11:45.8117442Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:11:45.8119073Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:11:45.8120745Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:11:45.8176338Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:11:45.8247320Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:11:45.8248985Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:11:45.8250374Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:11:45.8251592Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:11:45.8252843Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-06-26T07:11:45.8254163Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-06-26T07:11:45.8255716Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-06-26T07:11:45.8257374Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-06-26T07:11:45.8258972Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-06-26T07:11:45.8260472Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-06-26T07:11:45.8262028Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-06-26T07:11:45.8263889Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-06-26T07:11:45.8265714Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-06-26T07:11:45.8267269Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-06-26T07:11:45.8268822Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-06-26T07:11:45.8270475Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-06-26T07:11:45.8408254Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-06-26T07:11:45.8409847Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-06-26T07:11:45.8411806Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-06-26T07:11:45.8413548Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-06-26T07:11:45.8415283Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-06-26T07:11:45.8416839Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-06-26T07:11:45.8418470Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-06-26T07:11:45.8420108Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-06-26T07:11:45.8421744Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-06-26T07:11:45.8423377Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-06-26T07:11:45.8425209Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-06-26T07:11:45.8436671Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-06-26T07:11:45.8502431Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-06-26T07:11:45.8504386Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:11:45.8505935Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:11:45.8507283Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-06-26T07:11:45.8508561Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-06-26T07:11:45.8509824Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-06-26T07:11:45.8511071Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-06-26T07:11:45.8512237Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-06-26T07:11:45.8513300Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-06-26T07:11:45.8514450Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-06-26T07:11:45.8632876Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-06-26T07:11:45.8683072Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-06-26T07:11:45.8683862Z creating: build/lib/ 2024-06-26T07:11:45.8788611Z inflating: build/lib/libprotobuf-lite.a 2024-06-26T07:11:45.9335978Z inflating: build/lib/libprotobuf.a 2024-06-26T07:11:45.9346804Z inflating: build/lib/libpthreadpool.a 2024-06-26T07:11:45.9355981Z inflating: build/lib/libcpuinfo.a 2024-06-26T07:11:45.9365476Z inflating: build/lib/libcpuinfo_internals.a 2024-06-26T07:11:45.9366084Z inflating: build/lib/libclog.a 2024-06-26T07:11:45.9387687Z inflating: build/lib/libnnpack.a 2024-06-26T07:11:45.9389757Z inflating: build/lib/libnnpack_reference_layers.a 2024-06-26T07:11:45.9466996Z inflating: build/lib/libgtest.a 2024-06-26T07:11:45.9556790Z inflating: build/lib/libbenchmark.a 2024-06-26T07:11:45.9633157Z inflating: build/lib/libasmjit.a 2024-06-26T07:11:45.9641205Z inflating: build/lib/libittnotify.a 2024-06-26T07:11:45.9674096Z inflating: build/lib/libtensorpipe_uv.a 2024-06-26T07:11:45.9825963Z inflating: build/lib/libgloo.a 2024-06-26T07:11:45.9851493Z inflating: build/lib/libfmt.a 2024-06-26T07:11:45.9961532Z inflating: build/lib/libc10.so 2024-06-26T07:11:45.9962729Z inflating: build/lib/libcaffe2_nvrtc.so 2024-06-26T07:11:45.9963402Z inflating: build/lib/libfoxi_loader.a 2024-06-26T07:11:45.9965182Z inflating: build/lib/libtorch_global_deps.so 2024-06-26T07:11:45.9988167Z inflating: build/lib/libpytorch_qnnpack.a 2024-06-26T07:11:46.0590863Z inflating: build/lib/libprotoc.a 2024-06-26T07:11:46.0611502Z inflating: build/lib/libgmock.a 2024-06-26T07:11:46.0612048Z inflating: build/lib/libgtest_main.a 2024-06-26T07:11:46.0613239Z inflating: build/lib/libbenchmark_main.a 2024-06-26T07:11:46.1075839Z inflating: build/lib/libgloo_cuda.a 2024-06-26T07:11:47.3167673Z inflating: build/lib/libdnnl.a 2024-06-26T07:11:47.3848101Z inflating: build/lib/libtensorpipe.a 2024-06-26T07:11:47.3913883Z inflating: build/lib/libc10_cuda.so 2024-06-26T07:11:47.3914609Z inflating: build/lib/libgmock_main.a 2024-06-26T07:11:47.5407875Z inflating: build/lib/libfbgemm.a 2024-06-26T07:11:47.5711566Z inflating: build/lib/libtensorpipe_cuda.a 2024-06-26T07:11:47.6326499Z inflating: build/lib/libkineto.a 2024-06-26T07:11:47.6546900Z inflating: build/lib/libXNNPACK.a 2024-06-26T07:11:47.6597132Z inflating: build/lib/libonnx_proto.a 2024-06-26T07:11:47.7424879Z inflating: build/lib/libonnx.a 2024-06-26T07:11:50.6392018Z inflating: build/lib/libtorch_cpu.so 2024-06-26T07:11:50.6395800Z inflating: build/lib/libunbox_lib.a 2024-06-26T07:11:50.6400528Z inflating: build/lib/libshm.so 2024-06-26T07:11:53.1528754Z inflating: build/lib/libtorch_cuda.so 2024-06-26T07:11:53.1529420Z inflating: build/lib/libtorch.so 2024-06-26T07:11:53.1530393Z inflating: build/lib/libc10d_cuda_test.so 2024-06-26T07:11:54.1493610Z inflating: build/lib/libtorch_cuda_linalg.so 2024-06-26T07:11:54.3852696Z inflating: build/lib/libtorch_python.so 2024-06-26T07:11:54.3933649Z inflating: build/lib/libtorchbind_test.so 2024-06-26T07:11:54.3958345Z inflating: build/lib/libjitbackend_test.so 2024-06-26T07:11:54.3988706Z inflating: build/lib/libbackend_with_compiler.so 2024-06-26T07:11:54.4029827Z inflating: build/lib/libnnapi_backend.so 2024-06-26T07:11:54.4030396Z creating: build/bin/ 2024-06-26T07:11:54.4090427Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-06-26T07:11:54.4151852Z inflating: build/bin/c10_DeviceGuard_test 2024-06-26T07:11:54.4211816Z inflating: build/bin/c10_Device_test 2024-06-26T07:11:54.4282334Z inflating: build/bin/c10_DispatchKeySet_test 2024-06-26T07:11:54.4346562Z inflating: build/bin/c10_Scalar_test 2024-06-26T07:11:54.4404756Z inflating: build/bin/c10_StreamGuard_test 2024-06-26T07:11:54.4465446Z inflating: build/bin/c10_SymInt_test 2024-06-26T07:11:54.4530611Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-06-26T07:11:54.4597075Z inflating: build/bin/c10_InlineStreamGuard_test 2024-06-26T07:11:54.4664359Z inflating: build/bin/c10_SizesAndStrides_test 2024-06-26T07:11:54.4749329Z inflating: build/bin/c10_cow_test 2024-06-26T07:11:54.4812493Z inflating: build/bin/c10_Bitset_test 2024-06-26T07:11:54.4871086Z inflating: build/bin/c10_ConstexprCrc_test 2024-06-26T07:11:54.4930866Z inflating: build/bin/c10_DeadlockDetection_test 2024-06-26T07:11:54.4991308Z inflating: build/bin/c10_Half_test 2024-06-26T07:11:54.5059200Z inflating: build/bin/c10_LeftRight_test 2024-06-26T07:11:54.5125142Z inflating: build/bin/c10_Metaprogramming_test 2024-06-26T07:11:54.5184938Z inflating: build/bin/c10_Synchronized_test 2024-06-26T07:11:54.5251138Z inflating: build/bin/c10_ThreadLocal_test 2024-06-26T07:11:54.5313017Z inflating: build/bin/c10_TypeIndex_test 2024-06-26T07:11:54.5374153Z inflating: build/bin/c10_TypeList_test 2024-06-26T07:11:54.5433377Z inflating: build/bin/c10_TypeTraits_test 2024-06-26T07:11:54.5494535Z inflating: build/bin/c10_accumulate_test 2024-06-26T07:11:54.5561081Z inflating: build/bin/c10_bfloat16_test 2024-06-26T07:11:54.5620942Z inflating: build/bin/c10_bit_cast_test 2024-06-26T07:11:54.5688160Z inflating: build/bin/c10_complex_math_test 2024-06-26T07:11:54.5754885Z inflating: build/bin/c10_complex_test 2024-06-26T07:11:54.5817748Z inflating: build/bin/c10_exception_test 2024-06-26T07:11:54.5877691Z inflating: build/bin/c10_flags_test 2024-06-26T07:11:54.5937654Z inflating: build/bin/c10_generic_math_test 2024-06-26T07:11:54.6133537Z inflating: build/bin/c10_intrusive_ptr_test 2024-06-26T07:11:54.6194096Z inflating: build/bin/c10_irange_test 2024-06-26T07:11:54.6258098Z inflating: build/bin/c10_lazy_test 2024-06-26T07:11:54.6327511Z inflating: build/bin/c10_logging_test 2024-06-26T07:11:54.6416717Z inflating: build/bin/c10_optional_test 2024-06-26T07:11:54.6491259Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-06-26T07:11:54.6555487Z inflating: build/bin/c10_registry_test 2024-06-26T07:11:54.6736652Z inflating: build/bin/c10_small_vector_test 2024-06-26T07:11:54.6798618Z inflating: build/bin/c10_ssize_test 2024-06-26T07:11:54.6860558Z inflating: build/bin/c10_string_util_test 2024-06-26T07:11:54.6931179Z inflating: build/bin/c10_string_view_test 2024-06-26T07:11:54.6991395Z inflating: build/bin/c10_tempfile_test 2024-06-26T07:11:54.7058015Z inflating: build/bin/c10_typeid_test 2024-06-26T07:11:54.7116097Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-06-26T07:11:54.7653270Z inflating: build/bin/protoc-3.13.0.0 2024-06-26T07:11:54.8187864Z inflating: build/bin/protoc 2024-06-26T07:11:54.8250715Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-06-26T07:11:54.8313780Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-06-26T07:11:54.8376780Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-06-26T07:11:54.8438801Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-06-26T07:11:54.8502472Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-06-26T07:11:54.8565720Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-06-26T07:11:54.8623668Z inflating: build/bin/c10_cuda_CUDATest 2024-06-26T07:11:54.8687221Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-06-26T07:11:54.9081846Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-06-26T07:11:54.9493267Z inflating: build/bin/vec_test_all_types_AVX512 2024-06-26T07:11:54.9922234Z inflating: build/bin/vec_test_all_types_AVX2 2024-06-26T07:11:54.9985474Z inflating: build/bin/FileStoreTest 2024-06-26T07:11:55.0054205Z inflating: build/bin/TCPStoreTest 2024-06-26T07:11:55.0120870Z inflating: build/bin/HashStoreTest 2024-06-26T07:11:55.0137154Z inflating: build/bin/ProcessGroupMPITest 2024-06-26T07:11:55.0203286Z inflating: build/bin/test_edge_op_registration 2024-06-26T07:11:55.0208123Z inflating: build/bin/torch_shm_manager 2024-06-26T07:11:55.0211516Z inflating: build/bin/example_allreduce 2024-06-26T07:11:55.0279858Z inflating: build/bin/test_dist_autograd 2024-06-26T07:11:55.0363555Z inflating: build/bin/test_cpp_rpc 2024-06-26T07:11:55.0365742Z inflating: build/bin/parallel_benchmark 2024-06-26T07:11:55.0446085Z inflating: build/bin/test_mobile_nnc 2024-06-26T07:11:55.0456401Z inflating: build/bin/aot_model_compiler_test 2024-06-26T07:11:55.0869406Z inflating: build/bin/test_lazy 2024-06-26T07:11:55.2288840Z inflating: build/bin/test_api 2024-06-26T07:11:55.2375329Z inflating: build/bin/Dict_test 2024-06-26T07:11:55.2437336Z inflating: build/bin/Dimname_test 2024-06-26T07:11:55.2514652Z inflating: build/bin/MaybeOwned_test 2024-06-26T07:11:55.2582849Z inflating: build/bin/NamedTensor_test 2024-06-26T07:11:55.2653573Z inflating: build/bin/apply_utils_test 2024-06-26T07:11:55.2724139Z inflating: build/bin/atest 2024-06-26T07:11:55.2799726Z inflating: build/bin/basic 2024-06-26T07:11:55.2865317Z inflating: build/bin/broadcast_test 2024-06-26T07:11:55.2926977Z inflating: build/bin/cpu_allocator_test 2024-06-26T07:11:55.2996592Z inflating: build/bin/cpu_generator_test 2024-06-26T07:11:55.3059651Z inflating: build/bin/cpu_profiling_allocator_test 2024-06-26T07:11:55.3172173Z inflating: build/bin/cpu_rng_test 2024-06-26T07:11:55.3231688Z inflating: build/bin/dispatch_key_set_test 2024-06-26T07:11:55.3291612Z inflating: build/bin/dlconvertor_test 2024-06-26T07:11:55.3361520Z inflating: build/bin/extension_backend_test 2024-06-26T07:11:55.3427380Z inflating: build/bin/half_test 2024-06-26T07:11:55.3542056Z inflating: build/bin/ivalue_test 2024-06-26T07:11:55.3601599Z inflating: build/bin/lazy_tensor_test 2024-06-26T07:11:55.3666810Z inflating: build/bin/math_kernel_test 2024-06-26T07:11:55.3731516Z inflating: build/bin/memory_format_test 2024-06-26T07:11:55.3795036Z inflating: build/bin/memory_overlapping_test 2024-06-26T07:11:55.3858508Z inflating: build/bin/mobile_memory_cleanup 2024-06-26T07:11:55.3925591Z inflating: build/bin/native_test 2024-06-26T07:11:55.3985934Z inflating: build/bin/operator_name_test 2024-06-26T07:11:55.4048521Z inflating: build/bin/operators_test 2024-06-26T07:11:55.4110099Z inflating: build/bin/packedtensoraccessor_test 2024-06-26T07:11:55.4190441Z inflating: build/bin/pow_test 2024-06-26T07:11:55.4259975Z inflating: build/bin/quantized_test 2024-06-26T07:11:55.4320209Z inflating: build/bin/reduce_ops_test 2024-06-26T07:11:55.4381254Z inflating: build/bin/reportMemoryUsage_test 2024-06-26T07:11:55.4448917Z inflating: build/bin/scalar_tensor_test 2024-06-26T07:11:55.4519076Z inflating: build/bin/scalar_test 2024-06-26T07:11:55.4581090Z inflating: build/bin/StorageUtils_test 2024-06-26T07:11:55.4644058Z inflating: build/bin/stride_properties_test 2024-06-26T07:11:55.4738454Z inflating: build/bin/tensor_iterator_test 2024-06-26T07:11:55.4803040Z inflating: build/bin/test_parallel 2024-06-26T07:11:55.4805982Z inflating: build/bin/thread_init_test 2024-06-26T07:11:55.4872604Z inflating: build/bin/type_ptr_test 2024-06-26T07:11:55.4944470Z inflating: build/bin/type_test 2024-06-26T07:11:55.5006982Z inflating: build/bin/undefined_tensor_test 2024-06-26T07:11:55.5008693Z inflating: build/bin/verify_api_visibility 2024-06-26T07:11:55.5092176Z inflating: build/bin/legacy_vmap_test 2024-06-26T07:11:55.5153945Z inflating: build/bin/weakref_test 2024-06-26T07:11:55.5216013Z inflating: build/bin/wrapdim_test 2024-06-26T07:11:55.5277701Z inflating: build/bin/xla_tensor_test 2024-06-26T07:11:55.5349827Z inflating: build/bin/IListRef_test 2024-06-26T07:11:55.5476786Z inflating: build/bin/List_test 2024-06-26T07:11:55.5557234Z inflating: build/bin/KernelFunction_test 2024-06-26T07:11:55.5703286Z inflating: build/bin/kernel_function_legacy_test 2024-06-26T07:11:55.5819565Z inflating: build/bin/kernel_function_test 2024-06-26T07:11:55.5972790Z inflating: build/bin/kernel_lambda_legacy_test 2024-06-26T07:11:55.6097155Z inflating: build/bin/kernel_lambda_test 2024-06-26T07:11:55.6170121Z inflating: build/bin/kernel_stackbased_test 2024-06-26T07:11:55.6285448Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-06-26T07:11:55.6346618Z inflating: build/bin/CppSignature_test 2024-06-26T07:11:55.6405003Z inflating: build/bin/op_allowlist_test 2024-06-26T07:11:55.6471737Z inflating: build/bin/backend_fallback_test 2024-06-26T07:11:55.6834721Z inflating: build/bin/op_registration_test 2024-06-26T07:11:55.6911040Z inflating: build/bin/inline_container_test 2024-06-26T07:11:55.6973967Z inflating: build/bin/cuda_apply_test 2024-06-26T07:11:55.7034716Z inflating: build/bin/cuda_allocator_test 2024-06-26T07:11:55.7105539Z inflating: build/bin/cuda_atomic_ops_test 2024-06-26T07:11:55.7170537Z inflating: build/bin/cuda_caching_host_allocator_test 2024-06-26T07:11:55.7254487Z inflating: build/bin/cuda_complex_math_test 2024-06-26T07:11:55.7325266Z inflating: build/bin/cuda_complex_test 2024-06-26T07:11:55.7384596Z inflating: build/bin/cuda_device_test 2024-06-26T07:11:55.7453350Z inflating: build/bin/cuda_cub_test 2024-06-26T07:11:55.7514444Z inflating: build/bin/cuda_dlconvertor_test 2024-06-26T07:11:55.7592602Z inflating: build/bin/cuda_distributions_test 2024-06-26T07:11:55.7660517Z inflating: build/bin/cuda_generator_test 2024-06-26T07:11:55.7719875Z inflating: build/bin/cuda_half_test 2024-06-26T07:11:55.7780891Z inflating: build/bin/cuda_integer_divider_test 2024-06-26T07:11:55.7840006Z inflating: build/bin/cuda_optional_test 2024-06-26T07:11:55.7901425Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-06-26T07:11:55.7964392Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-06-26T07:11:55.8023901Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-06-26T07:11:55.8097450Z inflating: build/bin/cuda_stream_test 2024-06-26T07:11:55.8156470Z inflating: build/bin/cuda_cudnn_test 2024-06-26T07:11:55.8219035Z inflating: build/bin/cuda_vectorized_test 2024-06-26T07:11:55.8236454Z inflating: build/bin/tutorial_tensorexpr 2024-06-26T07:11:55.8315494Z inflating: build/bin/ProcessGroupGlooTest 2024-06-26T07:11:55.8384143Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-06-26T07:11:55.8459844Z inflating: build/bin/ProcessGroupNCCLTest 2024-06-26T07:11:55.8534184Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-06-26T07:11:55.9531450Z inflating: build/bin/test_tensorexpr 2024-06-26T07:11:56.0230079Z inflating: build/bin/test_jit 2024-06-26T07:11:56.0230682Z creating: .additional_ci_files/ 2024-06-26T07:11:56.0294950Z inflating: .additional_ci_files/test-times.json 2024-06-26T07:11:56.0554647Z inflating: .additional_ci_files/test-class-times.json 2024-06-26T07:11:56.0604446Z ##[group]Run rm artifacts.zip 2024-06-26T07:11:56.0604948Z rm artifacts.zip 2024-06-26T07:11:56.0618850Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:56.0619466Z env: 2024-06-26T07:11:56.0619792Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:56.0620343Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:56.0620933Z ##[endgroup] 2024-06-26T07:11:56.1837957Z ##[group]Run df -H 2024-06-26T07:11:56.1838354Z df -H 2024-06-26T07:11:56.1851799Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:56.1852408Z env: 2024-06-26T07:11:56.1852729Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:56.1853270Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:56.1853943Z ##[endgroup] 2024-06-26T07:11:56.1908937Z Filesystem Size Used Avail Use% Mounted on 2024-06-26T07:11:56.1910051Z /dev/root 1.1T 235G 832G 22% / 2024-06-26T07:11:56.1910746Z devtmpfs 45G 0 45G 0% /dev 2024-06-26T07:11:56.1911363Z tmpfs 45G 0 45G 0% /dev/shm 2024-06-26T07:11:56.1912204Z tmpfs 9.0G 1.1M 9.0G 1% /run 2024-06-26T07:11:56.1912906Z tmpfs 5.3M 0 5.3M 0% /run/lock 2024-06-26T07:11:56.1914093Z tmpfs 45G 0 45G 0% /sys/fs/cgroup 2024-06-26T07:11:56.1915087Z /dev/loop0 16M 16M 0 100% /snap/aws-cli/130 2024-06-26T07:11:56.1915762Z /dev/loop10 97M 97M 0 100% /snap/lxd/24061 2024-06-26T07:11:56.1916413Z /dev/loop9 97M 97M 0 100% /snap/lxd/23991 2024-06-26T07:11:56.1917012Z /dev/sda15 110M 6.4M 104M 6% /boot/efi 2024-06-26T07:11:56.1917638Z /dev/loop14 110M 110M 0 100% /snap/core/16928 2024-06-26T07:11:56.1918295Z /dev/loop7 68M 68M 0 100% /snap/core20/2264 2024-06-26T07:11:56.1918959Z /dev/loop15 41M 41M 0 100% /snap/snapd/21465 2024-06-26T07:11:56.1919608Z /dev/loop8 59M 59M 0 100% /snap/core18/2823 2024-06-26T07:11:56.1920263Z /dev/loop6 68M 68M 0 100% /snap/core20/2318 2024-06-26T07:11:56.1921136Z /dev/loop3 41M 41M 0 100% /snap/snapd/21759 2024-06-26T07:11:56.1921973Z /dev/loop11 371M 371M 0 100% /snap/google-cloud-sdk/460 2024-06-26T07:11:56.1922704Z /dev/loop1 59M 59M 0 100% /snap/core18/2829 2024-06-26T07:11:56.1923362Z /dev/loop4 110M 110M 0 100% /snap/core/17200 2024-06-26T07:11:56.1924246Z /dev/loop13 371M 371M 0 100% /snap/google-cloud-sdk/462 2024-06-26T07:11:56.2000968Z Prepare all required actions 2024-06-26T07:11:56.2001899Z Getting action download info 2024-06-26T07:11:56.3485877Z ##[group]Run ./.github/actions/download-td-artifacts 2024-06-26T07:11:56.3486442Z with: 2024-06-26T07:11:56.3486735Z env: 2024-06-26T07:11:56.3487051Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:56.3487595Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:56.3488165Z ##[endgroup] 2024-06-26T07:11:56.3529699Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:11:56.3530247Z with: 2024-06-26T07:11:56.3530555Z name: td_results 2024-06-26T07:11:56.3530924Z s3-bucket: gha-artifacts 2024-06-26T07:11:56.3531337Z region: us-east-1 2024-06-26T07:11:56.3531687Z env: 2024-06-26T07:11:56.3531981Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:56.3532514Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:56.3533091Z ##[endgroup] 2024-06-26T07:11:56.9363777Z (node:3596964) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:11:56.9364709Z 2024-06-26T07:11:56.9365036Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:11:56.9365925Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:11:56.9367119Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:11:57.0642234Z Found 0 objects with prefix pytorch/pytorch/9673646806/td_results/ 2024-06-26T07:11:57.0651449Z Artifact download has finished successfully 2024-06-26T07:11:57.0887924Z ##[group]Run mkdir -p .additional_ci_files 2024-06-26T07:11:57.0888531Z mkdir -p .additional_ci_files 2024-06-26T07:11:57.0889284Z mv td_results.json .additional_ci_files/td_results.json 2024-06-26T07:11:57.0902685Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:57.0903285Z env: 2024-06-26T07:11:57.0903609Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:57.0904553Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:57.0905199Z ##[endgroup] 2024-06-26T07:11:57.0973140Z mv: cannot stat 'td_results.json': No such file or directory 2024-06-26T07:11:57.0991789Z ##[error]Process completed with exit code 1. 2024-06-26T07:11:57.1034024Z ##[group]Run .github/scripts/parse_ref.py 2024-06-26T07:11:57.1034601Z .github/scripts/parse_ref.py 2024-06-26T07:11:57.1046571Z shell: /usr/bin/bash -e {0} 2024-06-26T07:11:57.1046987Z env: 2024-06-26T07:11:57.1047306Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:57.1047846Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:57.1048435Z ##[endgroup] 2024-06-26T07:11:57.1378843Z Prepare all required actions 2024-06-26T07:11:57.1429771Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-06-26T07:11:57.1430400Z with: 2024-06-26T07:11:57.1431376Z github-token: *** 2024-06-26T07:11:57.1431727Z env: 2024-06-26T07:11:57.1432020Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:57.1432558Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:57.1433140Z ##[endgroup] 2024-06-26T07:11:57.1453434Z ##[group]Run set -eux 2024-06-26T07:11:57.1453820Z set -eux 2024-06-26T07:11:57.1454538Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-06-26T07:11:57.1467877Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:11:57.1468493Z env: 2024-06-26T07:11:57.1468794Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:57.1469333Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:57.1470154Z GITHUB_TOKEN: *** 2024-06-26T07:11:57.1470743Z ##[endgroup] 2024-06-26T07:11:57.1507761Z + python3 .github/scripts/get_workflow_job_id.py 9673646806 gh-ci-gcp-a100-4 2024-06-26T07:11:57.8319870Z setting job-id=26688887958 2024-06-26T07:11:57.8321051Z setting job-name=cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-06-26T07:11:57.8598292Z Prepare all required actions 2024-06-26T07:11:57.8598866Z Getting action download info 2024-06-26T07:11:58.0017854Z ##[group]Run ./.github/actions/filter-test-configs 2024-06-26T07:11:58.0018427Z with: 2024-06-26T07:11:58.0019014Z github-token: *** 2024-06-26T07:11:58.0019950Z test-matrix: {"include": [{"config": "inductor_torchbench_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "linux.gcp.a100"}]} 2024-06-26T07:11:58.0021421Z job-name: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-06-26T07:11:58.0022367Z env: 2024-06-26T07:11:58.0022683Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:58.0023226Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:58.0023868Z ##[endgroup] 2024-06-26T07:11:58.0085332Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:11:58.0086044Z with: 2024-06-26T07:11:58.0086342Z shell: bash 2024-06-26T07:11:58.0086757Z timeout_minutes: 10 2024-06-26T07:11:58.0087127Z max_attempts: 5 2024-06-26T07:11:58.0087495Z retry_wait_seconds: 30 2024-06-26T07:11:58.0089142Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:11:58.0090654Z polling_interval_seconds: 1 2024-06-26T07:11:58.0091098Z warning_on_retry: true 2024-06-26T07:11:58.0091490Z continue_on_error: false 2024-06-26T07:11:58.0091888Z env: 2024-06-26T07:11:58.0092198Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:11:58.0092714Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:11:58.0093584Z GITHUB_TOKEN: *** 2024-06-26T07:11:58.0093950Z ##[endgroup] 2024-06-26T07:11:58.0765016Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:11:59.2475217Z Requirement already satisfied: requests==2.27.1 in /home/ubuntu/.local/lib/python3.8/site-packages (2.27.1) 2024-06-26T07:11:59.2597219Z Requirement already satisfied: pyyaml==6.0.1 in /home/ubuntu/.local/lib/python3.8/site-packages (6.0.1) 2024-06-26T07:11:59.2613590Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests==2.27.1) (1.25.8) 2024-06-26T07:11:59.2676907Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ubuntu/.local/lib/python3.8/site-packages (from requests==2.27.1) (2.0.12) 2024-06-26T07:11:59.2708682Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /usr/lib/python3/dist-packages (from requests==2.27.1) (2.8) 2024-06-26T07:11:59.2722949Z Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests==2.27.1) (2019.11.28) 2024-06-26T07:12:00.0782032Z Command completed after 1 attempt(s). 2024-06-26T07:12:00.0843637Z ##[group]Run set -x 2024-06-26T07:12:00.0844112Z set -x 2024-06-26T07:12:00.0844451Z  2024-06-26T07:12:00.0845132Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:12:00.0845957Z # in runner workspace 2024-06-26T07:12:00.0846589Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-06-26T07:12:00.0859922Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:00.0860537Z env: 2024-06-26T07:12:00.0860856Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:00.0861403Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:00.0861981Z ##[endgroup] 2024-06-26T07:12:00.0900814Z + python3 /home/weiwangmeta/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-06-26T07:12:00.1175912Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:12:00.1176561Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:12:00.1177119Z echo "Job name: ${JOB_NAME}" 2024-06-26T07:12:00.1177590Z  2024-06-26T07:12:00.1178495Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:12:00.1179402Z # in runner workspace 2024-06-26T07:12:00.1180103Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-06-26T07:12:00.1180893Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-06-26T07:12:00.1181424Z  --job-name "${JOB_NAME}" \ 2024-06-26T07:12:00.1182540Z  --test-matrix "{"include": [{"config": "inductor_torchbench_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "linux.gcp.a100"}]}" \ 2024-06-26T07:12:00.1183680Z  --selected-test-configs "" \ 2024-06-26T07:12:00.1184448Z  --pr-number "${PR_NUMBER}" \ 2024-06-26T07:12:00.1184964Z  --tag "${TAG}" \ 2024-06-26T07:12:00.1185408Z  --event-name "${EVENT_NAME}" \ 2024-06-26T07:12:00.1185940Z  --schedule "${SCHEDULE}" \ 2024-06-26T07:12:00.1186445Z  --branch "${HEAD_BRANCH}" 2024-06-26T07:12:00.1199344Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:00.1199937Z env: 2024-06-26T07:12:00.1200256Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:00.1200803Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:00.1201658Z GITHUB_TOKEN: *** 2024-06-26T07:12:00.1202452Z JOB_NAME: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-06-26T07:12:00.1203358Z PR_NUMBER: 2024-06-26T07:12:00.1203713Z TAG: ciflow/inductor/129470 2024-06-26T07:12:00.1204204Z EVENT_NAME: push 2024-06-26T07:12:00.1204541Z SCHEDULE: 2024-06-26T07:12:00.1204869Z HEAD_BRANCH: 2024-06-26T07:12:00.1205206Z ##[endgroup] 2024-06-26T07:12:00.1241259Z Workflow: inductor-periodic 2024-06-26T07:12:00.1242389Z Job name: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-06-26T07:12:00.4066910Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-06-26T07:12:00.7146704Z ##[group]Run echo "Filtered matrix:" 2024-06-26T07:12:00.7147272Z echo "Filtered matrix:" 2024-06-26T07:12:00.7148304Z echo "{"include": [{"config": "inductor_torchbench_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "linux.gcp.a100"}]}" 2024-06-26T07:12:00.7149322Z  2024-06-26T07:12:00.7149620Z echo 2024-06-26T07:12:00.7150063Z echo "Is the current job unstable? False" 2024-06-26T07:12:00.7150619Z  2024-06-26T07:12:00.7150923Z echo 2024-06-26T07:12:00.7151328Z echo "Is keep-going label set? False" 2024-06-26T07:12:00.7151857Z  2024-06-26T07:12:00.7152160Z echo 2024-06-26T07:12:00.7152537Z echo "Renabled issues? " 2024-06-26T07:12:00.7165585Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:00.7166203Z env: 2024-06-26T07:12:00.7166531Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:00.7167077Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:00.7167651Z ##[endgroup] 2024-06-26T07:12:00.7205023Z Filtered matrix: 2024-06-26T07:12:00.7205942Z {include: [{config: inductor_torchbench_smoketest_perf, shard: 1, num_shards: 1, runner: linux.gcp.a100}]} 2024-06-26T07:12:00.7207276Z 2024-06-26T07:12:00.7207577Z Is the current job unstable? False 2024-06-26T07:12:00.7208055Z 2024-06-26T07:12:00.7208480Z Is keep-going label set? False 2024-06-26T07:12:00.7208802Z 2024-06-26T07:12:00.7208940Z Renabled issues? 2024-06-26T07:12:00.7261238Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:12:00.7262126Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:12:00.7275041Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:00.7275869Z env: 2024-06-26T07:12:00.7276181Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:00.7276734Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:00.7277332Z JOB_TIMEOUT: 240 2024-06-26T07:12:00.7277679Z ##[endgroup] 2024-06-26T07:12:00.7403468Z ##[group]Run set -x 2024-06-26T07:12:00.7403925Z set -x 2024-06-26T07:12:00.7404248Z  2024-06-26T07:12:00.7404727Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-06-26T07:12:00.7405387Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-06-26T07:12:00.7406051Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-06-26T07:12:00.7406656Z  TEST_COMMAND=.ci/onnx/test.sh 2024-06-26T07:12:00.7407128Z else 2024-06-26T07:12:00.7407521Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:12:00.7408005Z fi 2024-06-26T07:12:00.7408299Z  2024-06-26T07:12:00.7408846Z # detached container should get cleaned up by teardown_ec2_linux 2024-06-26T07:12:00.7409808Z # TODO: Stop building test binaries as part of the build phase 2024-06-26T07:12:00.7410619Z # Used for GPU_FLAG since that doesn't play nice 2024-06-26T07:12:00.7411317Z # shellcheck disable=SC2086,SC2090 2024-06-26T07:12:00.7411871Z container_name=$(docker run \ 2024-06-26T07:12:00.7412357Z  ${GPU_FLAG:-} \ 2024-06-26T07:12:00.7412793Z  -e BUILD_ENVIRONMENT \ 2024-06-26T07:12:00.7413266Z  -e PR_NUMBER \ 2024-06-26T07:12:00.7413672Z  -e GITHUB_ACTIONS \ 2024-06-26T07:12:00.7414130Z  -e GITHUB_REPOSITORY \ 2024-06-26T07:12:00.7414663Z  -e GITHUB_WORKFLOW \ 2024-06-26T07:12:00.7415113Z  -e GITHUB_JOB \ 2024-06-26T07:12:00.7415521Z  -e GITHUB_RUN_ID \ 2024-06-26T07:12:00.7415973Z  -e GITHUB_RUN_NUMBER \ 2024-06-26T07:12:00.7416454Z  -e GITHUB_RUN_ATTEMPT \ 2024-06-26T07:12:00.7416928Z  -e JOB_ID \ 2024-06-26T07:12:00.7417302Z  -e JOB_NAME \ 2024-06-26T07:12:00.7417698Z  -e BASE_SHA \ 2024-06-26T07:12:00.7418094Z  -e BRANCH \ 2024-06-26T07:12:00.7418476Z  -e SHA1 \ 2024-06-26T07:12:00.7418859Z  -e AWS_DEFAULT_REGION \ 2024-06-26T07:12:00.7419334Z  -e IN_WHEEL_TEST \ 2024-06-26T07:12:00.7419799Z  -e SHARD_NUMBER \ 2024-06-26T07:12:00.7420217Z  -e TEST_CONFIG \ 2024-06-26T07:12:00.7420646Z  -e NUM_TEST_SHARDS \ 2024-06-26T07:12:00.7421105Z  -e REENABLED_ISSUES \ 2024-06-26T07:12:00.7421594Z  -e CONTINUE_THROUGH_ERROR \ 2024-06-26T07:12:00.7422084Z  -e VERBOSE_TEST_LOGS \ 2024-06-26T07:12:00.7422552Z  -e NO_TEST_TIMEOUT \ 2024-06-26T07:12:00.7422992Z  -e NO_TD \ 2024-06-26T07:12:00.7423382Z  -e TD_DISTRIBUTED \ 2024-06-26T07:12:00.7423816Z  -e PR_LABELS \ 2024-06-26T07:12:00.7424468Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-06-26T07:12:00.7425012Z  -e SCCACHE_BUCKET \ 2024-06-26T07:12:00.7425478Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-06-26T07:12:00.7425948Z  -e XLA_CUDA \ 2024-06-26T07:12:00.7426426Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-06-26T07:12:00.7427037Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-06-26T07:12:00.7427671Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-06-26T07:12:00.7428277Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-06-26T07:12:00.7428836Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-06-26T07:12:00.7429333Z  -e DASHBOARD_TAG \ 2024-06-26T07:12:00.7429903Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-06-26T07:12:00.7430559Z  --security-opt seccomp=unconfined \ 2024-06-26T07:12:00.7431116Z  --cap-add=SYS_PTRACE \ 2024-06-26T07:12:00.7431576Z  --ipc=host \ 2024-06-26T07:12:00.7431995Z  --shm-size="${SHM_SIZE}" \ 2024-06-26T07:12:00.7432694Z  --tty \ 2024-06-26T07:12:00.7433054Z  --detach \ 2024-06-26T07:12:00.7433469Z  --name="${container_name}" \ 2024-06-26T07:12:00.7433955Z  --user jenkins \ 2024-06-26T07:12:00.7434738Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-06-26T07:12:00.7435427Z  -w /var/lib/jenkins/workspace \ 2024-06-26T07:12:00.7435950Z  "${DOCKER_IMAGE}" 2024-06-26T07:12:00.7436345Z ) 2024-06-26T07:12:00.7436829Z # Propagate download.pytorch.org IP to container 2024-06-26T07:12:00.7437990Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-06-26T07:12:00.7439209Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-06-26T07:12:00.7440362Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-06-26T07:12:00.7452770Z shell: /usr/bin/bash -e {0} 2024-06-26T07:12:00.7453193Z env: 2024-06-26T07:12:00.7453510Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:00.7454058Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:00.7455068Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-06-26T07:12:00.7455683Z PR_NUMBER: 2024-06-26T07:12:00.7456061Z GITHUB_REPOSITORY: pytorch/pytorch 2024-06-26T07:12:00.7456575Z GITHUB_WORKFLOW: inductor-periodic 2024-06-26T07:12:00.7457038Z GITHUB_JOB: test 2024-06-26T07:12:00.7457400Z GITHUB_RUN_ID: 9673646806 2024-06-26T07:12:00.7457817Z GITHUB_RUN_NUMBER: 45567 2024-06-26T07:12:00.7458227Z GITHUB_RUN_ATTEMPT: 1 2024-06-26T07:12:00.7458598Z JOB_ID: 26688887958 2024-06-26T07:12:00.7459401Z JOB_NAME: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-06-26T07:12:00.7460295Z BRANCH: 2024-06-26T07:12:00.7460681Z SHA1: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:00.7461302Z BASE_SHA: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:00.7461935Z TEST_CONFIG: inductor_torchbench_smoketest_perf 2024-06-26T07:12:00.7462481Z SHARD_NUMBER: 1 2024-06-26T07:12:00.7462816Z NUM_TEST_SHARDS: 1 2024-06-26T07:12:00.7463194Z REENABLED_ISSUES: 2024-06-26T07:12:00.7463591Z CONTINUE_THROUGH_ERROR: False 2024-06-26T07:12:00.7464220Z VERBOSE_TEST_LOGS: False 2024-06-26T07:12:00.7465400Z NO_TEST_TIMEOUT: False 2024-06-26T07:12:00.7465875Z NO_TD: False 2024-06-26T07:12:00.7466229Z TD_DISTRIBUTED: False 2024-06-26T07:12:00.7466742Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-06-26T07:12:00.7467365Z SCCACHE_S3_KEY_PREFIX: inductor-periodic 2024-06-26T07:12:00.7467877Z SHM_SIZE: 2g 2024-06-26T07:12:00.7469327Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:12:00.7470891Z XLA_CUDA: 2024-06-26T07:12:00.7471518Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:12:00.7472325Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-06-26T07:12:00.7472851Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-06-26T07:12:00.7473335Z DASHBOARD_TAG: 2024-06-26T07:12:00.7474040Z HUGGING_FACE_HUB_TOKEN: *** 2024-06-26T07:12:00.7474473Z ##[endgroup] 2024-06-26T07:12:00.7512285Z + [[ inductor_torchbench_smoketest_perf == \m\u\l\t\i\g\p\u ]] 2024-06-26T07:12:00.7513868Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *onnx* ]] 2024-06-26T07:12:00.7514548Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:12:00.7524169Z +++ nproc --ignore=2 2024-06-26T07:12:00.7544987Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=10 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673646806 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/weiwangmeta/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:12:04.6714132Z + container_name=15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:12:04.6715634Z + grep download.pytorch.org /etc/hosts 2024-06-26T07:12:04.6718423Z + docker exec -i 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 sudo bash -c '/bin/cat >> /etc/hosts' 2024-06-26T07:12:04.7551620Z + echo DOCKER_CONTAINER_ID=15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:12:04.7556916Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:12:04.7559512Z + docker exec -t 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-06-26T07:12:05.3002055Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54) 2024-06-26T07:12:06.3686220Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1) 2024-06-26T07:12:06.3691341Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2) 2024-06-26T07:12:06.3696705Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1) 2024-06-26T07:12:06.3700014Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8) 2024-06-26T07:12:06.3705004Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4) 2024-06-26T07:12:06.3709503Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0) 2024-06-26T07:12:06.3744703Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0) 2024-06-26T07:12:06.3816415Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2) 2024-06-26T07:12:06.4382584Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5) 2024-06-26T07:12:06.4578575Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0) 2024-06-26T07:12:08.0434900Z Installing collected packages: torch 2024-06-26T07:12:20.1248761Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-06-26T07:12:20.1250880Z timm 0.9.7 requires torchvision, which is not installed. 2024-06-26T07:12:20.1252421Z Successfully installed torch-2.5.0a0+gitb8c4c54 2024-06-26T07:12:20.2383284Z ++ dirname .ci/pytorch/test.sh 2024-06-26T07:12:20.2391715Z + source .ci/pytorch/common.sh 2024-06-26T07:12:20.2397155Z +++ dirname .ci/pytorch/common.sh 2024-06-26T07:12:20.2408902Z ++ source .ci/pytorch/common_utils.sh 2024-06-26T07:12:20.2410395Z +++ declare -f -t trap_add 2024-06-26T07:12:20.2418495Z ++ set -ex 2024-06-26T07:12:20.2419508Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *rocm* ]] 2024-06-26T07:12:20.2420592Z ++ BUILD_TEST_LIBTORCH=0 2024-06-26T07:12:20.2421790Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 != *rocm* ]] 2024-06-26T07:12:20.2425871Z ++ stat -c %u /var/lib/jenkins/workspace 2024-06-26T07:12:20.2441985Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-06-26T07:12:20.2442891Z + trap_add cleanup_workspace EXIT 2024-06-26T07:12:20.2443606Z + trap_add_cmd=cleanup_workspace 2024-06-26T07:12:20.2444105Z + shift 2024-06-26T07:12:20.2444424Z + for trap_add_name in "$@" 2024-06-26T07:12:20.2452422Z +++ trap -p EXIT 2024-06-26T07:12:20.2456721Z ++ eval 'extract_trap_cmd ' 2024-06-26T07:12:20.2457611Z +++ extract_trap_cmd 2024-06-26T07:12:20.2458395Z +++ printf '%s\n' '' 2024-06-26T07:12:20.2459125Z ++ printf '%s\n' cleanup_workspace 2024-06-26T07:12:20.2460888Z + trap -- ' 2024-06-26T07:12:20.2461708Z cleanup_workspace' EXIT 2024-06-26T07:12:20.2462554Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-06-26T07:12:20.8453506Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-06-26T07:12:20.8487464Z + echo 'Environment variables:' 2024-06-26T07:12:20.8488092Z Environment variables: 2024-06-26T07:12:20.8488474Z + env 2024-06-26T07:12:20.8499527Z INSTALLED_DB=yes 2024-06-26T07:12:20.8501345Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:12:20.8502805Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:12:20.8503745Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:12:20.8505396Z GITHUB_WORKSPACE=/home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-06-26T07:12:20.8507572Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:12:20.8508590Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:12:20.8509819Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:12:20.8510901Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-06-26T07:12:20.8511530Z HOSTNAME=15cecb048bf7 2024-06-26T07:12:20.8512652Z GITHUB_PATH=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/add_path_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8513715Z GITHUB_ACTION=__self 2024-06-26T07:12:20.8514194Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:12:20.8519280Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:12:20.8524761Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:12:20.8525380Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:12:20.8525781Z GITHUB_RUN_NUMBER=45567 2024-06-26T07:12:20.8526252Z TEST_CONFIG=inductor_torchbench_smoketest_perf 2024-06-26T07:12:20.8526820Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:12:20.8527396Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:12:20.8527938Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:12:20.8528467Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:12:20.8528975Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:12:20.8529538Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:12:20.8530521Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:12:20.8531086Z GITHUB_REF_TYPE=tag 2024-06-26T07:12:20.8531467Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:12:20.8531904Z NCCL_VERSION=2.17.1-1 2024-06-26T07:12:20.8532513Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:20.8533051Z XLA_CUDA= 2024-06-26T07:12:20.8534440Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:12:20.8537629Z *** 2024-06-26T07:12:20.8537958Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:12:20.8538434Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:12:20.8538860Z GITHUB_ACTIONS=true 2024-06-26T07:12:20.8539273Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:20.8539928Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:12:20.8540590Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:12:20.8541156Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:20.8541761Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:12:20.8542340Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:20.8543543Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:12:20.8544906Z UCC_HOME=/usr 2024-06-26T07:12:20.8545303Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:12:20.8545798Z VERBOSE_TEST_LOGS=False 2024-06-26T07:12:20.8546199Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:12:20.8546757Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:12:20.8547341Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:12:20.8547905Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:12:20.8548349Z SHARD_NUMBER=1 2024-06-26T07:12:20.8548705Z GITHUB_REF_PROTECTED=false 2024-06-26T07:12:20.8549098Z HOME=/var/lib/jenkins 2024-06-26T07:12:20.8549523Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:12:20.8550053Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:12:20.8550615Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:12:20.8551290Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-06-26T07:12:20.8551801Z CUDA_VERSION=12.1.1 2024-06-26T07:12:20.8552307Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:12:20.8552818Z NUM_TEST_SHARDS=1 2024-06-26T07:12:20.8553162Z UCX_HOME=/usr 2024-06-26T07:12:20.8553804Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:12:20.8555283Z GITHUB_STATE=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/save_state_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8556875Z JOB_NAME=cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-06-26T07:12:20.8558486Z GITHUB_ENV=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/set_env_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8559983Z GITHUB_EVENT_PATH=/home/weiwangmeta/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:12:20.8560825Z GITHUB_EVENT_NAME=push 2024-06-26T07:12:20.8561191Z DASHBOARD_TAG= 2024-06-26T07:12:20.8561540Z GITHUB_RUN_ID=9673646806 2024-06-26T07:12:20.8562103Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:12:20.8562757Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:12:20.8564100Z GITHUB_STEP_SUMMARY=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/step_summary_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8565290Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:12:20.8565775Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:12:20.8566216Z PR_NUMBER= 2024-06-26T07:12:20.8566523Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:12:20.8566915Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:12:20.8567443Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:12:20.8567985Z TERM=xterm 2024-06-26T07:12:20.8568372Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:12:20.8568861Z INSTALLED_VISION=yes 2024-06-26T07:12:20.8569211Z BRANCH= 2024-06-26T07:12:20.8569518Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:12:20.8569990Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:12:20.8570768Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:12:20.8571756Z GITHUB_ACTION_PATH=/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:12:20.8572707Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:12:20.8573407Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:12:20.8574024Z REENABLED_ISSUES= 2024-06-26T07:12:20.8574351Z SHLVL=1 2024-06-26T07:12:20.8574645Z MAX_JOBS=10 2024-06-26T07:12:20.8575009Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:12:20.8575425Z NVARCH=x86_64 2024-06-26T07:12:20.8575750Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:12:20.8576289Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:20.8576938Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:12:20.8577529Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:12:20.8578345Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:12:20.8579061Z GITHUB_JOB=test 2024-06-26T07:12:20.8579533Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:12:20.8580210Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:12:20.8580813Z NO_TEST_TIMEOUT=False 2024-06-26T07:12:20.8581188Z TD_DISTRIBUTED=False 2024-06-26T07:12:20.8581625Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:12:20.8582136Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:12:20.8582637Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:12:20.8583060Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:12:20.8583462Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:12:20.8583893Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:12:20.8585689Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:20.8587049Z GITHUB_BASE_REF= 2024-06-26T07:12:20.8587425Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:12:20.8587839Z CI=true 2024-06-26T07:12:20.8588213Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:12:20.8588702Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:12:20.8589141Z JOB_ID=26688887958 2024-06-26T07:12:20.8589499Z INSTALLED_PROTOBUF=yes 2024-06-26T07:12:20.8589879Z GITHUB_HEAD_REF= 2024-06-26T07:12:20.8590227Z GITHUB_ACTION_REF= 2024-06-26T07:12:20.8590740Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:12:20.8591390Z GITHUB_WORKFLOW=inductor-periodic 2024-06-26T07:12:20.8591865Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:12:20.8593052Z GITHUB_OUTPUT=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/set_output_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8594149Z NO_TD=False 2024-06-26T07:12:20.8594496Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:12:20.8594920Z _=/usr/bin/env 2024-06-26T07:12:20.8595497Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-06-26T07:12:20.8717465Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-06-26T07:12:20.8719623Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-06-26T07:12:20.8721123Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-06-26T07:12:20.8722300Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-06-26T07:12:20.8723039Z + BUILD_DIR=build 2024-06-26T07:12:20.8723435Z + BUILD_RENAMED_DIR=build_renamed 2024-06-26T07:12:20.8724049Z + BUILD_BIN_DIR=build/bin 2024-06-26T07:12:20.8724452Z + SHARD_NUMBER=1 2024-06-26T07:12:20.8724786Z + NUM_TEST_SHARDS=1 2024-06-26T07:12:20.8725153Z + export VALGRIND=ON 2024-06-26T07:12:20.8725525Z + VALGRIND=ON 2024-06-26T07:12:20.8726085Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *clang9* ]] 2024-06-26T07:12:20.8727192Z + [[ 0 == \1 ]] 2024-06-26T07:12:20.8727643Z + [[ False == \1 ]] 2024-06-26T07:12:20.8728247Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 != *bazel* ]] 2024-06-26T07:12:20.8728869Z ++ realpath build/custom_test_artifacts 2024-06-26T07:12:20.8739396Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-06-26T07:12:20.8741352Z + [[ -n '' ]] 2024-06-26T07:12:20.8741841Z + echo 'Environment variables' 2024-06-26T07:12:20.8742408Z Environment variables 2024-06-26T07:12:20.8742771Z + env 2024-06-26T07:12:20.8750714Z INSTALLED_DB=yes 2024-06-26T07:12:20.8751622Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-06-26T07:12:20.8752783Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:12:20.8753809Z NV_NVML_DEV_VERSION=12.1.105-1 2024-06-26T07:12:20.8755270Z GITHUB_WORKSPACE=/home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-06-26T07:12:20.8756191Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:12:20.8756803Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-06-26T07:12:20.8757425Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:12:20.8758190Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-06-26T07:12:20.8759177Z HOSTNAME=15cecb048bf7 2024-06-26T07:12:20.8761134Z GITHUB_PATH=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/add_path_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8762903Z GITHUB_ACTION=__self 2024-06-26T07:12:20.8763398Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:12:20.8768505Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-06-26T07:12:20.8773693Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-06-26T07:12:20.8774371Z NV_NVTX_VERSION=12.1.105-1 2024-06-26T07:12:20.8774793Z GITHUB_RUN_NUMBER=45567 2024-06-26T07:12:20.8775248Z TEST_CONFIG=inductor_torchbench_smoketest_perf 2024-06-26T07:12:20.8775811Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:12:20.8776382Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:12:20.8776928Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-06-26T07:12:20.8777458Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-06-26T07:12:20.8777967Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-06-26T07:12:20.8778506Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:12:20.8779094Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:12:20.8779652Z GITHUB_REF_TYPE=tag 2024-06-26T07:12:20.8780031Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:12:20.8780473Z NCCL_VERSION=2.17.1-1 2024-06-26T07:12:20.8780916Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:20.8781449Z XLA_CUDA= 2024-06-26T07:12:20.8782064Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:12:20.8782542Z *** 2024-06-26T07:12:20.8782846Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:12:20.8783321Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:12:20.8783751Z GITHUB_ACTIONS=true 2024-06-26T07:12:20.8784620Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:12:20.8785228Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-06-26T07:12:20.8785898Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-06-26T07:12:20.8786458Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:20.8787062Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:12:20.8787639Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:20.8788829Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:12:20.8789796Z UCC_HOME=/usr 2024-06-26T07:12:20.8790177Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-06-26T07:12:20.8790647Z VERBOSE_TEST_LOGS=False 2024-06-26T07:12:20.8791046Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:12:20.8791604Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-06-26T07:12:20.8792450Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:12:20.8793021Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-06-26T07:12:20.8793469Z SHARD_NUMBER=1 2024-06-26T07:12:20.8793821Z GITHUB_REF_PROTECTED=false 2024-06-26T07:12:20.8794269Z HOME=/var/lib/jenkins 2024-06-26T07:12:20.8794834Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:12:20.8795372Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:12:20.8795935Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:12:20.8796618Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-06-26T07:12:20.8797113Z CUDA_VERSION=12.1.1 2024-06-26T07:12:20.8797622Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-06-26T07:12:20.8798137Z NUM_TEST_SHARDS=1 2024-06-26T07:12:20.8798481Z UCX_HOME=/usr 2024-06-26T07:12:20.8799116Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-06-26T07:12:20.8800540Z GITHUB_STATE=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/save_state_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8802153Z JOB_NAME=cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-06-26T07:12:20.8803754Z GITHUB_ENV=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/set_env_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8805307Z GITHUB_EVENT_PATH=/home/weiwangmeta/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:12:20.8806142Z GITHUB_EVENT_NAME=push 2024-06-26T07:12:20.8806513Z DASHBOARD_TAG= 2024-06-26T07:12:20.8806859Z GITHUB_RUN_ID=9673646806 2024-06-26T07:12:20.8807409Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-06-26T07:12:20.8808068Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-06-26T07:12:20.8809367Z GITHUB_STEP_SUMMARY=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/step_summary_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8810560Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:12:20.8811044Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-06-26T07:12:20.8811485Z PR_NUMBER= 2024-06-26T07:12:20.8811797Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:12:20.8812158Z VALGRIND=ON 2024-06-26T07:12:20.8812494Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:12:20.8813022Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:12:20.8813566Z TERM=xterm 2024-06-26T07:12:20.8814001Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-06-26T07:12:20.8814499Z INSTALLED_VISION=yes 2024-06-26T07:12:20.8814846Z BRANCH= 2024-06-26T07:12:20.8815167Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:12:20.8815636Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:12:20.8816140Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:12:20.8817101Z GITHUB_ACTION_PATH=/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:12:20.8818062Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:12:20.8818654Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:12:20.8819219Z REENABLED_ISSUES= 2024-06-26T07:12:20.8819551Z SHLVL=1 2024-06-26T07:12:20.8819847Z MAX_JOBS=10 2024-06-26T07:12:20.8820215Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-06-26T07:12:20.8820613Z NVARCH=x86_64 2024-06-26T07:12:20.8820957Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:12:20.8821498Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:12:20.8822141Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:12:20.8822730Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-06-26T07:12:20.8823562Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:12:20.8824616Z GITHUB_JOB=test 2024-06-26T07:12:20.8825114Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-06-26T07:12:20.8825778Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:12:20.8826403Z NO_TEST_TIMEOUT=False 2024-06-26T07:12:20.8826786Z TD_DISTRIBUTED=False 2024-06-26T07:12:20.8827213Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-06-26T07:12:20.8827733Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:12:20.8828235Z NV_NVPROF_VERSION=12.1.105-1 2024-06-26T07:12:20.8828667Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:12:20.8829347Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:12:20.8829760Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:12:20.8831103Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:20.8832594Z GITHUB_BASE_REF= 2024-06-26T07:12:20.8832967Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:12:20.8833404Z CI=true 2024-06-26T07:12:20.8833783Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-06-26T07:12:20.8834321Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:12:20.8834751Z JOB_ID=26688887958 2024-06-26T07:12:20.8835116Z INSTALLED_PROTOBUF=yes 2024-06-26T07:12:20.8835501Z GITHUB_HEAD_REF= 2024-06-26T07:12:20.8835836Z GITHUB_ACTION_REF= 2024-06-26T07:12:20.8836370Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:12:20.8837016Z GITHUB_WORKFLOW=inductor-periodic 2024-06-26T07:12:20.8837499Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:12:20.8838690Z GITHUB_OUTPUT=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/set_output_252d8106-10bf-48e3-97eb-8af8ac52778b 2024-06-26T07:12:20.8839781Z NO_TD=False 2024-06-26T07:12:20.8840129Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:12:20.8840562Z _=/usr/bin/env 2024-06-26T07:12:20.8840978Z + echo 'Testing pytorch' 2024-06-26T07:12:20.8841389Z Testing pytorch 2024-06-26T07:12:20.8841764Z + export LANG=C.UTF-8 2024-06-26T07:12:20.8842155Z + LANG=C.UTF-8 2024-06-26T07:12:20.8842472Z + PR_NUMBER= 2024-06-26T07:12:20.8842945Z + [[ inductor_torchbench_smoketest_perf == \d\e\f\a\u\l\t ]] 2024-06-26T07:12:20.8843749Z + [[ inductor_torchbench_smoketest_perf == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-06-26T07:12:20.8844532Z + [[ inductor_torchbench_smoketest_perf == \s\l\o\w ]] 2024-06-26T07:12:20.8845394Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *slow-gradcheck* ]] 2024-06-26T07:12:20.8846268Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *cuda* ]] 2024-06-26T07:12:20.8846932Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:12:20.8847494Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:12:20.8848111Z + [[ inductor_torchbench_smoketest_perf == *crossref* ]] 2024-06-26T07:12:20.8848903Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *rocm* ]] 2024-06-26T07:12:20.8849696Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *xpu* ]] 2024-06-26T07:12:20.8850511Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 != *-bazel-* ]] 2024-06-26T07:12:20.8851204Z + pip_install --user ninja==1.10.2 2024-06-26T07:12:20.8851856Z + pip install --progress-bar off --user ninja==1.10.2 2024-06-26T07:12:21.4193176Z Collecting ninja==1.10.2 2024-06-26T07:12:21.5527524Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-06-26T07:12:21.6103279Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-06-26T07:12:23.2726285Z Installing collected packages: ninja 2024-06-26T07:12:23.2821346Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-06-26T07:12:23.2823008Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-06-26T07:12:23.2875073Z Successfully installed ninja-1.10.2 2024-06-26T07:12:23.3955732Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:23.3958602Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:23.3960633Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *aarch64* ]] 2024-06-26T07:12:23.3961263Z + install_tlparse 2024-06-26T07:12:23.3961702Z + pip_install --user tlparse==0.3.7 2024-06-26T07:12:23.3962381Z + pip install --progress-bar off --user tlparse==0.3.7 2024-06-26T07:12:23.9307985Z Collecting tlparse==0.3.7 2024-06-26T07:12:24.0746995Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-06-26T07:12:24.1222496Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-06-26T07:12:25.9319955Z Installing collected packages: tlparse 2024-06-26T07:12:25.9824474Z Successfully installed tlparse-0.3.7 2024-06-26T07:12:26.0831514Z ++ python -m site --user-base 2024-06-26T07:12:26.1067345Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:12:26.1069748Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *asan* ]] 2024-06-26T07:12:26.1071065Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *-debug* ]] 2024-06-26T07:12:26.1072529Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 != *-bazel-* ]] 2024-06-26T07:12:26.1073823Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm80. Expect the assertion to pass' 2024-06-26T07:12:26.1075348Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm80. Expect the assertion to pass 2024-06-26T07:12:26.1076231Z + cd test 2024-06-26T07:12:26.1076890Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-06-26T07:12:28.1713872Z + [[ inductor_torchbench_smoketest_perf == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-06-26T07:12:28.1715226Z + [[ inductor_torchbench_smoketest_perf == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-06-26T07:12:28.1716578Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 != *-bazel-* ]] 2024-06-26T07:12:28.1717189Z + pushd test 2024-06-26T07:12:28.1717731Z ~/workspace/test ~/workspace 2024-06-26T07:12:28.1718384Z ++ python -c 'import torch; print(torch.version.cuda)' 2024-06-26T07:12:30.1316496Z + CUDA_VERSION=12.1 2024-06-26T07:12:30.1318369Z + '[' 12.1 == 12.4 ']' 2024-06-26T07:12:30.1319039Z + ISCUDA124= 2024-06-26T07:12:30.1319542Z + popd 2024-06-26T07:12:30.1320079Z ~/workspace 2024-06-26T07:12:30.1320513Z + DYNAMO_BENCHMARK_FLAGS=() 2024-06-26T07:12:30.1321583Z + [[ inductor_torchbench_smoketest_perf == *dynamo_eager* ]] 2024-06-26T07:12:30.1322946Z + [[ inductor_torchbench_smoketest_perf == *aot_eager* ]] 2024-06-26T07:12:30.1323823Z + [[ inductor_torchbench_smoketest_perf == *aot_inductor* ]] 2024-06-26T07:12:30.1324560Z + [[ inductor_torchbench_smoketest_perf == *inductor* ]] 2024-06-26T07:12:30.1325249Z + [[ inductor_torchbench_smoketest_perf != *perf* ]] 2024-06-26T07:12:30.1325962Z + [[ inductor_torchbench_smoketest_perf == *dynamic* ]] 2024-06-26T07:12:30.1326672Z + [[ inductor_torchbench_smoketest_perf == *cpu_inductor* ]] 2024-06-26T07:12:30.1327504Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-06-26T07:12:30.1358971Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *libtorch* ]] 2024-06-26T07:12:30.1359963Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *-bazel-* ]] 2024-06-26T07:12:30.1363088Z + cd test 2024-06-26T07:12:30.1363784Z + python -c 'import torch; print(torch.__config__.show())' 2024-06-26T07:12:31.9892210Z PyTorch built with: 2024-06-26T07:12:31.9892940Z - GCC 9.4 2024-06-26T07:12:31.9893335Z - C++ Version: 201703 2024-06-26T07:12:31.9894562Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:12:31.9895944Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:12:31.9896784Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:12:31.9898329Z - LAPACK is enabled (usually provided by MKL) 2024-06-26T07:12:31.9899366Z - NNPACK is enabled 2024-06-26T07:12:31.9899845Z - CPU capability usage: AVX512 2024-06-26T07:12:31.9900367Z - CUDA Runtime 12.1 2024-06-26T07:12:31.9901024Z - NVCC architecture flags: -gencode;arch=compute_80,code=sm_80 2024-06-26T07:12:31.9901944Z - CuDNN 90.1 (built against CUDA 12.4) 2024-06-26T07:12:31.9902467Z - Magma 2.6.1 2024-06-26T07:12:31.9913030Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-06-26T07:12:31.9922407Z 2024-06-26T07:12:32.3996467Z + cd test 2024-06-26T07:12:32.3997491Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-06-26T07:12:34.0979928Z ATen/Parallel: 2024-06-26T07:12:34.0981682Z at::get_num_threads() : 6 2024-06-26T07:12:34.0982448Z at::get_num_interop_threads() : 6 2024-06-26T07:12:34.0983317Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:12:34.0984522Z omp_get_max_threads() : 6 2024-06-26T07:12:34.0986637Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:12:34.0988404Z mkl_get_max_threads() : 6 2024-06-26T07:12:34.0989747Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:12:34.0991051Z std::thread::hardware_concurrency() : 12 2024-06-26T07:12:34.0991984Z Environment variables: 2024-06-26T07:12:34.0992694Z OMP_NUM_THREADS : [not set] 2024-06-26T07:12:34.0993475Z MKL_NUM_THREADS : [not set] 2024-06-26T07:12:34.0994269Z ATen parallel backend: OpenMP 2024-06-26T07:12:34.0994754Z 2024-06-26T07:12:34.3832680Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *aarch64* ]] 2024-06-26T07:12:34.3833824Z + [[ inductor_torchbench_smoketest_perf == *backward* ]] 2024-06-26T07:12:34.3834535Z + [[ inductor_torchbench_smoketest_perf == *xla* ]] 2024-06-26T07:12:34.3835231Z + [[ inductor_torchbench_smoketest_perf == *executorch* ]] 2024-06-26T07:12:34.3836000Z + [[ inductor_torchbench_smoketest_perf == \j\i\t\_\l\e\g\a\c\y ]] 2024-06-26T07:12:34.3836917Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *libtorch* ]] 2024-06-26T07:12:34.3837654Z + [[ inductor_torchbench_smoketest_perf == distributed ]] 2024-06-26T07:12:34.3838441Z + [[ inductor_torchbench_smoketest_perf == *inductor_distributed* ]] 2024-06-26T07:12:34.3839438Z + [[ inductor_torchbench_smoketest_perf == *inductor-halide* ]] 2024-06-26T07:12:34.3840410Z + [[ inductor_torchbench_smoketest_perf == *inductor-micro-benchmark* ]] 2024-06-26T07:12:34.3841231Z + [[ inductor_torchbench_smoketest_perf == *huggingface* ]] 2024-06-26T07:12:34.3841934Z + [[ inductor_torchbench_smoketest_perf == *timm* ]] 2024-06-26T07:12:34.3842648Z + [[ inductor_torchbench_smoketest_perf == *torchbench* ]] 2024-06-26T07:12:34.3843383Z + [[ inductor_torchbench_smoketest_perf == *cpu_inductor* ]] 2024-06-26T07:12:34.3843987Z + install_torchaudio cuda 2024-06-26T07:12:34.3844389Z + local commit 2024-06-26T07:12:34.3844745Z ++ get_pinned_commit audio 2024-06-26T07:12:34.3845189Z ++ cat .github/ci_commit_pins/audio.txt 2024-06-26T07:12:34.3859333Z + commit=b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:12:34.3860268Z + [[ cuda == \c\u\d\a ]] 2024-06-26T07:12:34.3861057Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-06-26T07:12:34.3862248Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:12:34.3864834Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:12:34.8343583Z Collecting git+https://github.com/pytorch/audio.git@b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:12:34.8351372Z Cloning https://github.com/pytorch/audio.git (to revision b829e936f7cc61b48149f5f957a451a38bf2a178) to /tmp/pip-req-build-93lm5doz 2024-06-26T07:12:34.8382428Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio.git /tmp/pip-req-build-93lm5doz 2024-06-26T07:12:36.0475781Z Running command git rev-parse -q --verify 'sha^b829e936f7cc61b48149f5f957a451a38bf2a178' 2024-06-26T07:12:36.0503112Z Running command git fetch -q https://github.com/pytorch/audio.git b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:12:36.7047628Z Running command git checkout -q b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:12:37.0111957Z Resolved https://github.com/pytorch/audio.git to commit b829e936f7cc61b48149f5f957a451a38bf2a178 2024-06-26T07:12:37.0113390Z Running command git submodule update --init --recursive -q 2024-06-26T07:12:39.9112440Z Preparing metadata (setup.py) ... [?25l- done 2024-06-26T07:12:39.9155128Z [?25hRequirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchaudio==2.4.0a0+b829e93) (2.5.0a0+gitb8c4c54) 2024-06-26T07:12:39.9234812Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (3.13.1) 2024-06-26T07:12:39.9241819Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (4.12.2) 2024-06-26T07:12:39.9245667Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (1.12.1) 2024-06-26T07:12:39.9249395Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (2.8.8) 2024-06-26T07:12:39.9253973Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (3.1.4) 2024-06-26T07:12:39.9257033Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b829e93) (2024.2.0) 2024-06-26T07:12:39.9786569Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchaudio==2.4.0a0+b829e93) (2.1.5) 2024-06-26T07:12:39.9979449Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchaudio==2.4.0a0+b829e93) (1.3.0) 2024-06-26T07:12:40.0068498Z Building wheels for collected packages: torchaudio 2024-06-26T07:16:18.2046633Z Building wheel for torchaudio (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-06-26T07:16:18.2125589Z [?25h Created wheel for torchaudio: filename=torchaudio-2.4.0a0+b829e93-cp310-cp310-linux_x86_64.whl size=2308821 sha256=895d637f718cf307ca2ec4b1c9fe0fab0a9ce1205a8a4618b5bc2e10942b827d 2024-06-26T07:16:18.2127981Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/30/b3/9c/882f183f00af2e582825654a92a04400924519922db53ff522 2024-06-26T07:16:18.2164899Z Successfully built torchaudio 2024-06-26T07:16:19.7362642Z Installing collected packages: torchaudio 2024-06-26T07:16:20.0018094Z Successfully installed torchaudio-2.4.0a0+b829e93 2024-06-26T07:16:20.1767495Z + install_torchtext 2024-06-26T07:16:20.1768074Z + local data_commit 2024-06-26T07:16:20.1768641Z + local text_commit 2024-06-26T07:16:20.1776970Z ++ get_pinned_commit data 2024-06-26T07:16:20.1777600Z ++ cat .github/ci_commit_pins/data.txt 2024-06-26T07:16:20.1800901Z + data_commit=11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:16:20.1805483Z ++ get_pinned_commit text 2024-06-26T07:16:20.1805939Z ++ cat .github/ci_commit_pins/text.txt 2024-06-26T07:16:20.1822080Z + text_commit=b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:16:20.1824191Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:16:20.1826100Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:16:20.6538552Z Collecting git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:16:20.6545630Z Cloning https://github.com/pytorch/data.git (to revision 11bb5b847ea8b9e0d9bb82db3304daf383008d3f) to /tmp/pip-req-build-bbvbmfd4 2024-06-26T07:16:20.6579384Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/data.git /tmp/pip-req-build-bbvbmfd4 2024-06-26T07:16:21.5843506Z Running command git rev-parse -q --verify 'sha^11bb5b847ea8b9e0d9bb82db3304daf383008d3f' 2024-06-26T07:16:21.5870755Z Running command git fetch -q https://github.com/pytorch/data.git 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:16:22.0051639Z Running command git checkout -q 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:16:22.3205094Z Resolved https://github.com/pytorch/data.git to commit 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-06-26T07:16:22.3207498Z Running command git submodule update --init --recursive -q 2024-06-26T07:17:52.8226087Z Preparing metadata (setup.py) ... [?25l- done 2024-06-26T07:17:52.8290097Z [?25hRequirement already satisfied: urllib3>=1.25 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (1.26.19) 2024-06-26T07:17:52.8294852Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.32.3) 2024-06-26T07:17:52.8302094Z Requirement already satisfied: torch>2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.5.0a0+gitb8c4c54) 2024-06-26T07:17:52.8378984Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.13.1) 2024-06-26T07:17:52.8385822Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (4.12.2) 2024-06-26T07:17:52.8390738Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.12.1) 2024-06-26T07:17:52.8395135Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.8.8) 2024-06-26T07:17:52.8398993Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.1.4) 2024-06-26T07:17:52.8403013Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2024.2.0) 2024-06-26T07:17:52.8631736Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.3.2) 2024-06-26T07:17:52.8640132Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.7) 2024-06-26T07:17:52.8648582Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (2024.6.2) 2024-06-26T07:17:52.9218105Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.1.5) 2024-06-26T07:17:52.9425423Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.3.0) 2024-06-26T07:17:52.9524900Z Building wheels for collected packages: torchdata 2024-06-26T07:17:55.3046124Z Building wheel for torchdata (setup.py) ... [?25l- \ | done 2024-06-26T07:17:55.3060256Z [?25h Created wheel for torchdata: filename=torchdata-0.7.0a0+11bb5b8-py3-none-any.whl size=182956 sha256=0973be298f44d2f2eafeacb7b51202bcde3c7663ba44c428f0a8348e2384656c 2024-06-26T07:17:55.3062365Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/ca/59/a3/c8250bfc8d3d4d639498d4beb2e0f0e70b9a508ac61fde85ce 2024-06-26T07:17:55.3096801Z Successfully built torchdata 2024-06-26T07:17:56.8036231Z Installing collected packages: torchdata 2024-06-26T07:17:56.9113331Z Successfully installed torchdata-0.7.0a0+11bb5b8 2024-06-26T07:17:59.9141752Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:17:59.9143752Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:18:00.3617889Z Collecting git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:18:00.3625545Z Cloning https://github.com/pytorch/text.git (to revision b0ebddc648d279826089db91775375221777a2db) to /tmp/pip-req-build-4cjhzcgy 2024-06-26T07:18:00.3652157Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/text.git /tmp/pip-req-build-4cjhzcgy 2024-06-26T07:18:01.6143945Z Running command git rev-parse -q --verify 'sha^b0ebddc648d279826089db91775375221777a2db' 2024-06-26T07:18:01.6171379Z Running command git fetch -q https://github.com/pytorch/text.git b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:18:02.3627191Z Running command git checkout -q b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:18:02.7119865Z Resolved https://github.com/pytorch/text.git to commit b0ebddc648d279826089db91775375221777a2db 2024-06-26T07:18:02.7121322Z Running command git submodule update --init --recursive -q 2024-06-26T07:18:11.7362929Z Preparing metadata (setup.py) ... [?25l- done 2024-06-26T07:18:11.7432284Z [?25hRequirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (4.66.4) 2024-06-26T07:18:11.7436844Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.32.3) 2024-06-26T07:18:11.7441297Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.5.0a0+gitb8c4c54) 2024-06-26T07:18:11.7447072Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (1.21.2) 2024-06-26T07:18:11.7451180Z Requirement already satisfied: torchdata in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (0.7.0a0+11bb5b8) 2024-06-26T07:18:11.7529033Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.3.2) 2024-06-26T07:18:11.7536156Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.7) 2024-06-26T07:18:11.7545788Z Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (1.26.19) 2024-06-26T07:18:11.7553255Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (2024.6.2) 2024-06-26T07:18:11.7624960Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.13.1) 2024-06-26T07:18:11.7634434Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (4.12.2) 2024-06-26T07:18:11.7656959Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (1.12.1) 2024-06-26T07:18:11.7661132Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2.8.8) 2024-06-26T07:18:11.7663318Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.1.4) 2024-06-26T07:18:11.7665853Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2024.2.0) 2024-06-26T07:18:11.8492786Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchtext==0.16.0a0+b0ebddc) (2.1.5) 2024-06-26T07:18:11.8702255Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchtext==0.16.0a0+b0ebddc) (1.3.0) 2024-06-26T07:18:11.8806064Z Building wheels for collected packages: torchtext 2024-06-26T07:19:08.3198453Z Building wheel for torchtext (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-06-26T07:19:08.3270071Z [?25h Created wheel for torchtext: filename=torchtext-0.16.0a0+b0ebddc-cp310-cp310-linux_x86_64.whl size=2045304 sha256=aed5237c8324585ffe1a5b89e7c067ef22af4229124cace7e797462df2b15fb3 2024-06-26T07:19:08.3272727Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/e5/3d/d7/faafad098ec9437ecdf4495c0f5e72b817fffbb06063e32cbf 2024-06-26T07:19:08.3304386Z Successfully built torchtext 2024-06-26T07:19:09.7657195Z Installing collected packages: torchtext 2024-06-26T07:19:09.9007040Z Successfully installed torchtext-0.16.0a0+b0ebddc 2024-06-26T07:19:10.0717558Z + install_torchvision 2024-06-26T07:19:10.0718066Z + local orig_preload 2024-06-26T07:19:10.0718451Z + local commit 2024-06-26T07:19:10.0724644Z ++ get_pinned_commit vision 2024-06-26T07:19:10.0725755Z ++ cat .github/ci_commit_pins/vision.txt 2024-06-26T07:19:10.0748168Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:19:10.0749172Z + orig_preload= 2024-06-26T07:19:10.0749835Z + '[' -n '' ']' 2024-06-26T07:19:10.0751330Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:19:10.0753509Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:19:10.5286023Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:19:10.5292540Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-ysrjs35x 2024-06-26T07:19:10.5325058Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-ysrjs35x 2024-06-26T07:19:12.5825019Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-06-26T07:19:12.5855480Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:19:14.3455110Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:19:14.8044778Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:19:17.9115829Z Preparing metadata (setup.py) ... [?25l- \ done 2024-06-26T07:19:17.9185038Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-06-26T07:19:17.9189388Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb8c4c54) 2024-06-26T07:19:17.9198990Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-06-26T07:19:17.9451028Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-06-26T07:19:17.9461694Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-06-26T07:19:17.9466323Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1) 2024-06-26T07:19:17.9472865Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-06-26T07:19:17.9477373Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-06-26T07:19:17.9483943Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.2.0) 2024-06-26T07:19:18.0008637Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-06-26T07:19:18.0208990Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-06-26T07:19:18.0303732Z Building wheels for collected packages: torchvision 2024-06-26T07:21:07.6992868Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-06-26T07:21:07.7067778Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2112968 sha256=214c5aa7a844363bd1ef78b07ff4b6817a9fb67c917f55a3ae50c134af6aee6d 2024-06-26T07:21:07.7070042Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-06-26T07:21:07.7119338Z Successfully built torchvision 2024-06-26T07:21:09.1945312Z Installing collected packages: torchvision 2024-06-26T07:21:09.6971530Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-06-26T07:21:09.8471704Z + '[' -n '' ']' 2024-06-26T07:21:09.8472135Z + id=0 2024-06-26T07:21:09.8472578Z + pip_install opencv-python==4.8.0.74 2024-06-26T07:21:09.8473340Z + pip install --progress-bar off opencv-python==4.8.0.74 2024-06-26T07:21:10.4924314Z Collecting opencv-python==4.8.0.74 2024-06-26T07:21:10.6231723Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB) 2024-06-26T07:21:10.6483815Z Requirement already satisfied: numpy>=1.21.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opencv-python==4.8.0.74) (1.21.2) 2024-06-26T07:21:10.6918511Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB) 2024-06-26T07:21:13.6008994Z Installing collected packages: opencv-python 2024-06-26T07:21:14.7542678Z Successfully installed opencv-python-4.8.0.74 2024-06-26T07:21:14.8754922Z + [[ inductor_torchbench_smoketest_perf == *inductor_torchbench_smoketest_perf* ]] 2024-06-26T07:21:14.8756033Z + checkout_install_torchbench hf_Bert hf_Albert nanogpt timm_vision_transformer 2024-06-26T07:21:14.8756784Z + local commit 2024-06-26T07:21:14.8760202Z ++ get_pinned_commit torchbench 2024-06-26T07:21:14.8761422Z ++ cat .github/ci_commit_pins/torchbench.txt 2024-06-26T07:21:14.8782156Z + commit=23512dbebd44a11eb84afbf53c3c071dd105297e 2024-06-26T07:21:14.8783446Z + git clone https://github.com/pytorch/benchmark torchbench 2024-06-26T07:21:14.8806732Z Cloning into 'torchbench'... 2024-06-26T07:21:15.2354250Z remote: Enumerating objects: 30187, done. 2024-06-26T07:21:15.2354987Z remote: Counting objects: 0% (1/546) 2024-06-26T07:21:15.2355638Z remote: Counting objects: 1% (6/546) 2024-06-26T07:21:15.2356241Z remote: Counting objects: 2% (11/546) 2024-06-26T07:21:15.2356860Z remote: Counting objects: 3% (17/546) 2024-06-26T07:21:15.2357473Z remote: Counting objects: 4% (22/546) 2024-06-26T07:21:15.2358109Z remote: Counting objects: 5% (28/546) 2024-06-26T07:21:15.2358702Z remote: Counting objects: 6% (33/546) 2024-06-26T07:21:15.2359416Z remote: Counting objects: 7% (39/546) 2024-06-26T07:21:15.2360194Z remote: Counting objects: 8% (44/546) 2024-06-26T07:21:15.2360853Z remote: Counting objects: 9% (50/546) 2024-06-26T07:21:15.2361656Z remote: Counting objects: 10% (55/546) 2024-06-26T07:21:15.2362267Z remote: Counting objects: 11% (61/546) 2024-06-26T07:21:15.2362876Z remote: Counting objects: 12% (66/546) 2024-06-26T07:21:15.2363608Z remote: Counting objects: 13% (71/546) 2024-06-26T07:21:15.2364643Z remote: Counting objects: 14% (77/546) 2024-06-26T07:21:15.2365851Z remote: Counting objects: 15% (82/546) 2024-06-26T07:21:15.2366860Z remote: Counting objects: 16% (88/546) 2024-06-26T07:21:15.2367474Z remote: Counting objects: 17% (93/546) 2024-06-26T07:21:15.2368602Z remote: Counting objects: 18% (99/546) 2024-06-26T07:21:15.2370005Z remote: Counting objects: 19% (104/546) 2024-06-26T07:21:15.2371248Z remote: Counting objects: 20% (110/546) 2024-06-26T07:21:15.2372395Z remote: Counting objects: 21% (115/546) 2024-06-26T07:21:15.2373464Z remote: Counting objects: 22% (121/546) 2024-06-26T07:21:15.2374692Z remote: Counting objects: 23% (126/546) 2024-06-26T07:21:15.2375752Z remote: Counting objects: 24% (132/546) 2024-06-26T07:21:15.2376990Z remote: Counting objects: 25% (137/546) 2024-06-26T07:21:15.2378010Z remote: Counting objects: 26% (142/546) 2024-06-26T07:21:15.2379253Z remote: Counting objects: 27% (148/546) 2024-06-26T07:21:15.2380516Z remote: Counting objects: 28% (153/546) 2024-06-26T07:21:15.2381234Z remote: Counting objects: 29% (159/546) 2024-06-26T07:21:15.2381843Z remote: Counting objects: 30% (164/546) 2024-06-26T07:21:15.2382459Z remote: Counting objects: 31% (170/546) 2024-06-26T07:21:15.2383079Z remote: Counting objects: 32% (175/546) 2024-06-26T07:21:15.2383714Z remote: Counting objects: 33% (181/546) 2024-06-26T07:21:15.2384639Z remote: Counting objects: 34% (186/546) 2024-06-26T07:21:15.2385255Z remote: Counting objects: 35% (192/546) 2024-06-26T07:21:15.2385876Z remote: Counting objects: 36% (197/546) 2024-06-26T07:21:15.2386509Z remote: Counting objects: 37% (203/546) 2024-06-26T07:21:15.2387122Z remote: Counting objects: 38% (208/546) 2024-06-26T07:21:15.2387737Z remote: Counting objects: 39% (213/546) 2024-06-26T07:21:15.2388358Z remote: Counting objects: 40% (219/546) 2024-06-26T07:21:15.2388976Z remote: Counting objects: 41% (224/546) 2024-06-26T07:21:15.2389635Z remote: Counting objects: 42% (230/546) 2024-06-26T07:21:15.2390256Z remote: Counting objects: 43% (235/546) 2024-06-26T07:21:15.2390874Z remote: Counting objects: 44% (241/546) 2024-06-26T07:21:15.2391475Z remote: Counting objects: 45% (246/546) 2024-06-26T07:21:15.2392092Z remote: Counting objects: 46% (252/546) 2024-06-26T07:21:15.2393082Z remote: Counting objects: 47% (257/546) 2024-06-26T07:21:15.2393696Z remote: Counting objects: 48% (263/546) 2024-06-26T07:21:15.2394298Z remote: Counting objects: 49% (268/546) 2024-06-26T07:21:15.2394909Z remote: Counting objects: 50% (273/546) 2024-06-26T07:21:15.2395677Z remote: Counting objects: 51% (279/546) 2024-06-26T07:21:15.2396304Z remote: Counting objects: 52% (284/546) 2024-06-26T07:21:15.2396908Z remote: Counting objects: 53% (290/546) 2024-06-26T07:21:15.2397523Z remote: Counting objects: 54% (295/546) 2024-06-26T07:21:15.2398145Z remote: Counting objects: 55% (301/546) 2024-06-26T07:21:15.2398763Z remote: Counting objects: 56% (306/546) 2024-06-26T07:21:15.2399410Z remote: Counting objects: 57% (312/546) 2024-06-26T07:21:15.2400027Z remote: Counting objects: 58% (317/546) 2024-06-26T07:21:15.2400646Z remote: Counting objects: 59% (323/546) 2024-06-26T07:21:15.2401259Z remote: Counting objects: 60% (328/546) 2024-06-26T07:21:15.2401873Z remote: Counting objects: 61% (334/546) 2024-06-26T07:21:15.2402491Z remote: Counting objects: 62% (339/546) 2024-06-26T07:21:15.2403105Z remote: Counting objects: 63% (344/546) 2024-06-26T07:21:15.2403705Z remote: Counting objects: 64% (350/546) 2024-06-26T07:21:15.2404330Z remote: Counting objects: 65% (355/546) 2024-06-26T07:21:15.2404941Z remote: Counting objects: 66% (361/546) 2024-06-26T07:21:15.2405563Z remote: Counting objects: 67% (366/546) 2024-06-26T07:21:15.2406166Z remote: Counting objects: 68% (372/546) 2024-06-26T07:21:15.2406781Z remote: Counting objects: 69% (377/546) 2024-06-26T07:21:15.2407396Z remote: Counting objects: 70% (383/546) 2024-06-26T07:21:15.2408010Z remote: Counting objects: 71% (388/546) 2024-06-26T07:21:15.2408658Z remote: Counting objects: 72% (394/546) 2024-06-26T07:21:15.2409647Z remote: Counting objects: 73% (399/546) 2024-06-26T07:21:15.2410387Z remote: Counting objects: 74% (405/546) 2024-06-26T07:21:15.2411568Z remote: Counting objects: 75% (410/546) 2024-06-26T07:21:15.2412184Z remote: Counting objects: 76% (415/546) 2024-06-26T07:21:15.2412797Z remote: Counting objects: 77% (421/546) 2024-06-26T07:21:15.2413416Z remote: Counting objects: 78% (426/546) 2024-06-26T07:21:15.2414041Z remote: Counting objects: 79% (432/546) 2024-06-26T07:21:15.2414645Z remote: Counting objects: 80% (437/546) 2024-06-26T07:21:15.2415260Z remote: Counting objects: 81% (443/546) 2024-06-26T07:21:15.2415921Z remote: Counting objects: 82% (448/546) 2024-06-26T07:21:15.2416665Z remote: Counting objects: 83% (454/546) 2024-06-26T07:21:15.2417702Z remote: Counting objects: 84% (459/546) 2024-06-26T07:21:15.2418561Z remote: Counting objects: 85% (465/546) 2024-06-26T07:21:15.2419278Z remote: Counting objects: 86% (470/546) 2024-06-26T07:21:15.2419895Z remote: Counting objects: 87% (476/546) 2024-06-26T07:21:15.2420498Z remote: Counting objects: 88% (481/546) 2024-06-26T07:21:15.2421334Z remote: Counting objects: 89% (486/546) 2024-06-26T07:21:15.2422059Z remote: Counting objects: 90% (492/546) 2024-06-26T07:21:15.2423249Z remote: Counting objects: 91% (497/546) 2024-06-26T07:21:15.2424104Z remote: Counting objects: 92% (503/546) 2024-06-26T07:21:15.2424729Z remote: Counting objects: 93% (508/546) 2024-06-26T07:21:15.2425344Z remote: Counting objects: 94% (514/546) 2024-06-26T07:21:15.2425959Z remote: Counting objects: 95% (519/546) 2024-06-26T07:21:15.2426753Z remote: Counting objects: 96% (525/546) 2024-06-26T07:21:15.2427732Z remote: Counting objects: 97% (530/546) 2024-06-26T07:21:15.2428765Z remote: Counting objects: 98% (536/546) 2024-06-26T07:21:15.2429420Z remote: Counting objects: 99% (541/546) 2024-06-26T07:21:15.2430033Z remote: Counting objects: 100% (546/546) 2024-06-26T07:21:15.2430705Z remote: Counting objects: 100% (546/546), done. 2024-06-26T07:21:15.2431390Z remote: Compressing objects: 0% (1/318) 2024-06-26T07:21:15.2432606Z remote: Compressing objects: 1% (4/318) 2024-06-26T07:21:15.2433755Z remote: Compressing objects: 2% (7/318) 2024-06-26T07:21:15.2434415Z remote: Compressing objects: 3% (10/318) 2024-06-26T07:21:15.2435072Z remote: Compressing objects: 4% (13/318) 2024-06-26T07:21:15.2435877Z remote: Compressing objects: 5% (16/318) 2024-06-26T07:21:15.2436656Z remote: Compressing objects: 6% (20/318) 2024-06-26T07:21:15.2437479Z remote: Compressing objects: 7% (23/318) 2024-06-26T07:21:15.2438648Z remote: Compressing objects: 8% (26/318) 2024-06-26T07:21:15.2439356Z remote: Compressing objects: 9% (29/318) 2024-06-26T07:21:15.2440003Z remote: Compressing objects: 10% (32/318) 2024-06-26T07:21:15.2440685Z remote: Compressing objects: 11% (35/318) 2024-06-26T07:21:15.2441327Z remote: Compressing objects: 12% (39/318) 2024-06-26T07:21:15.2442137Z remote: Compressing objects: 13% (42/318) 2024-06-26T07:21:15.2442787Z remote: Compressing objects: 14% (45/318) 2024-06-26T07:21:15.2443774Z remote: Compressing objects: 15% (48/318) 2024-06-26T07:21:15.2445026Z remote: Compressing objects: 16% (51/318) 2024-06-26T07:21:15.2446289Z remote: Compressing objects: 17% (55/318) 2024-06-26T07:21:15.2447551Z remote: Compressing objects: 18% (58/318) 2024-06-26T07:21:15.2448815Z remote: Compressing objects: 19% (61/318) 2024-06-26T07:21:15.2449596Z remote: Compressing objects: 20% (64/318) 2024-06-26T07:21:15.2450249Z remote: Compressing objects: 21% (67/318) 2024-06-26T07:21:15.2450943Z remote: Compressing objects: 22% (70/318) 2024-06-26T07:21:15.2452168Z remote: Compressing objects: 23% (74/318) 2024-06-26T07:21:15.2453219Z remote: Compressing objects: 24% (77/318) 2024-06-26T07:21:15.2454174Z remote: Compressing objects: 25% (80/318) 2024-06-26T07:21:15.2455228Z remote: Compressing objects: 26% (83/318) 2024-06-26T07:21:15.2455877Z remote: Compressing objects: 27% (86/318) 2024-06-26T07:21:15.2456507Z remote: Compressing objects: 28% (90/318) 2024-06-26T07:21:15.2457285Z remote: Compressing objects: 29% (93/318) 2024-06-26T07:21:15.2458385Z remote: Compressing objects: 30% (96/318) 2024-06-26T07:21:15.2459282Z remote: Compressing objects: 31% (99/318) 2024-06-26T07:21:15.2459943Z remote: Compressing objects: 32% (102/318) 2024-06-26T07:21:15.2460634Z remote: Compressing objects: 33% (105/318) 2024-06-26T07:21:15.2461296Z remote: Compressing objects: 34% (109/318) 2024-06-26T07:21:15.2461952Z remote: Compressing objects: 35% (112/318) 2024-06-26T07:21:15.2462781Z remote: Compressing objects: 36% (115/318) 2024-06-26T07:21:15.2463754Z remote: Compressing objects: 37% (118/318) 2024-06-26T07:21:15.2464977Z remote: Compressing objects: 38% (121/318) 2024-06-26T07:21:15.2465633Z remote: Compressing objects: 39% (125/318) 2024-06-26T07:21:15.2466276Z remote: Compressing objects: 40% (128/318) 2024-06-26T07:21:15.2466931Z remote: Compressing objects: 41% (131/318) 2024-06-26T07:21:15.2467592Z remote: Compressing objects: 42% (134/318) 2024-06-26T07:21:15.2468232Z remote: Compressing objects: 43% (137/318) 2024-06-26T07:21:15.2468890Z remote: Compressing objects: 44% (140/318) 2024-06-26T07:21:15.2469628Z remote: Compressing objects: 45% (144/318) 2024-06-26T07:21:15.2470286Z remote: Compressing objects: 46% (147/318) 2024-06-26T07:21:15.2470934Z remote: Compressing objects: 47% (150/318) 2024-06-26T07:21:15.2471575Z remote: Compressing objects: 48% (153/318) 2024-06-26T07:21:15.2472231Z remote: Compressing objects: 49% (156/318) 2024-06-26T07:21:15.2472881Z remote: Compressing objects: 50% (159/318) 2024-06-26T07:21:15.2473531Z remote: Compressing objects: 51% (163/318) 2024-06-26T07:21:15.2474171Z remote: Compressing objects: 52% (166/318) 2024-06-26T07:21:15.2474822Z remote: Compressing objects: 53% (169/318) 2024-06-26T07:21:15.2475473Z remote: Compressing objects: 54% (172/318) 2024-06-26T07:21:15.2476120Z remote: Compressing objects: 55% (175/318) 2024-06-26T07:21:15.2477015Z remote: Compressing objects: 56% (179/318) 2024-06-26T07:21:15.2477670Z remote: Compressing objects: 57% (182/318) 2024-06-26T07:21:15.2478325Z remote: Compressing objects: 58% (185/318) 2024-06-26T07:21:15.2479171Z remote: Compressing objects: 59% (188/318) 2024-06-26T07:21:15.2479839Z remote: Compressing objects: 60% (191/318) 2024-06-26T07:21:15.2480490Z remote: Compressing objects: 61% (194/318) 2024-06-26T07:21:15.2481139Z remote: Compressing objects: 62% (198/318) 2024-06-26T07:21:15.2481773Z remote: Compressing objects: 63% (201/318) 2024-06-26T07:21:15.2482451Z remote: Compressing objects: 64% (204/318) 2024-06-26T07:21:15.2483100Z remote: Compressing objects: 65% (207/318) 2024-06-26T07:21:15.2483746Z remote: Compressing objects: 66% (210/318) 2024-06-26T07:21:15.2484382Z remote: Compressing objects: 67% (214/318) 2024-06-26T07:21:15.2485037Z remote: Compressing objects: 68% (217/318) 2024-06-26T07:21:15.2485693Z remote: Compressing objects: 69% (220/318) 2024-06-26T07:21:15.2486348Z remote: Compressing objects: 70% (223/318) 2024-06-26T07:21:15.2486980Z remote: Compressing objects: 71% (226/318) 2024-06-26T07:21:15.2487645Z remote: Compressing objects: 72% (229/318) 2024-06-26T07:21:15.2488302Z remote: Compressing objects: 73% (233/318) 2024-06-26T07:21:15.2488951Z remote: Compressing objects: 74% (236/318) 2024-06-26T07:21:15.2489631Z remote: Compressing objects: 75% (239/318) 2024-06-26T07:21:15.2490284Z remote: Compressing objects: 76% (242/318) 2024-06-26T07:21:15.2490936Z remote: Compressing objects: 77% (245/318) 2024-06-26T07:21:15.2491588Z remote: Compressing objects: 78% (249/318) 2024-06-26T07:21:15.2492417Z remote: Compressing objects: 79% (252/318) 2024-06-26T07:21:15.2493082Z remote: Compressing objects: 80% (255/318) 2024-06-26T07:21:15.2493734Z remote: Compressing objects: 81% (258/318) 2024-06-26T07:21:15.2494377Z remote: Compressing objects: 82% (261/318) 2024-06-26T07:21:15.2495028Z remote: Compressing objects: 83% (264/318) 2024-06-26T07:21:15.2495680Z remote: Compressing objects: 84% (268/318) 2024-06-26T07:21:15.2496333Z remote: Compressing objects: 85% (271/318) 2024-06-26T07:21:15.2496982Z remote: Compressing objects: 86% (274/318) 2024-06-26T07:21:15.2497638Z remote: Compressing objects: 87% (277/318) 2024-06-26T07:21:15.2498294Z remote: Compressing objects: 88% (280/318) 2024-06-26T07:21:15.2498949Z remote: Compressing objects: 89% (284/318) 2024-06-26T07:21:15.2499637Z remote: Compressing objects: 90% (287/318) 2024-06-26T07:21:15.2500288Z remote: Compressing objects: 91% (290/318) 2024-06-26T07:21:15.2500946Z remote: Compressing objects: 92% (293/318) 2024-06-26T07:21:15.2501758Z remote: Compressing objects: 93% (296/318) 2024-06-26T07:21:15.2502394Z remote: Compressing objects: 94% (299/318) 2024-06-26T07:21:15.2503045Z remote: Compressing objects: 95% (303/318) 2024-06-26T07:21:15.2503704Z remote: Compressing objects: 96% (306/318) 2024-06-26T07:21:15.2504642Z remote: Compressing objects: 97% (309/318) 2024-06-26T07:21:15.2505286Z remote: Compressing objects: 98% (312/318) 2024-06-26T07:21:15.2505944Z remote: Compressing objects: 99% (315/318) 2024-06-26T07:21:15.2506596Z remote: Compressing objects: 100% (318/318) 2024-06-26T07:21:15.2507296Z remote: Compressing objects: 100% (318/318), done. 2024-06-26T07:21:15.2686468Z Receiving objects: 0% (1/30187) 2024-06-26T07:21:15.2743238Z Receiving objects: 1% (302/30187) 2024-06-26T07:21:15.2913377Z Receiving objects: 2% (604/30187) 2024-06-26T07:21:15.2955061Z Receiving objects: 3% (906/30187) 2024-06-26T07:21:15.2990109Z Receiving objects: 4% (1208/30187) 2024-06-26T07:21:15.3027237Z Receiving objects: 5% (1510/30187) 2024-06-26T07:21:15.3063712Z Receiving objects: 6% (1812/30187) 2024-06-26T07:21:15.3111496Z Receiving objects: 7% (2114/30187) 2024-06-26T07:21:15.3182032Z Receiving objects: 8% (2415/30187) 2024-06-26T07:21:15.3216785Z Receiving objects: 9% (2717/30187) 2024-06-26T07:21:15.3259559Z Receiving objects: 10% (3019/30187) 2024-06-26T07:21:15.3333455Z Receiving objects: 11% (3321/30187) 2024-06-26T07:21:15.3409740Z Receiving objects: 12% (3623/30187) 2024-06-26T07:21:15.3477733Z Receiving objects: 13% (3925/30187) 2024-06-26T07:21:15.3683845Z Receiving objects: 14% (4227/30187) 2024-06-26T07:21:15.3805370Z Receiving objects: 15% (4529/30187) 2024-06-26T07:21:15.3910875Z Receiving objects: 16% (4830/30187) 2024-06-26T07:21:15.3974881Z Receiving objects: 17% (5132/30187) 2024-06-26T07:21:15.4018543Z Receiving objects: 18% (5434/30187) 2024-06-26T07:21:15.4053227Z Receiving objects: 19% (5736/30187) 2024-06-26T07:21:15.4132424Z Receiving objects: 20% (6038/30187) 2024-06-26T07:21:15.4153805Z Receiving objects: 21% (6340/30187) 2024-06-26T07:21:15.4175305Z Receiving objects: 22% (6642/30187) 2024-06-26T07:21:15.5105731Z Receiving objects: 23% (6944/30187) 2024-06-26T07:21:16.2492052Z Receiving objects: 24% (7245/30187) 2024-06-26T07:21:16.5470858Z Receiving objects: 24% (7437/30187), 66.13 MiB | 66.19 MiB/s 2024-06-26T07:21:17.1291856Z Receiving objects: 25% (7547/30187), 66.13 MiB | 66.19 MiB/s 2024-06-26T07:21:17.2056799Z Receiving objects: 26% (7849/30187), 102.46 MiB | 68.35 MiB/s 2024-06-26T07:21:17.2494787Z Receiving objects: 27% (8151/30187), 102.46 MiB | 68.35 MiB/s 2024-06-26T07:21:17.2815283Z Receiving objects: 27% (8325/30187), 134.09 MiB | 67.08 MiB/s 2024-06-26T07:21:17.3672185Z Receiving objects: 28% (8453/30187), 134.09 MiB | 67.08 MiB/s 2024-06-26T07:21:17.4424817Z Receiving objects: 29% (8755/30187), 134.09 MiB | 67.08 MiB/s 2024-06-26T07:21:17.5181340Z Receiving objects: 30% (9057/30187), 134.09 MiB | 67.08 MiB/s 2024-06-26T07:21:17.5952032Z Receiving objects: 31% (9358/30187), 134.09 MiB | 67.08 MiB/s 2024-06-26T07:21:17.6708180Z Receiving objects: 32% (9660/30187), 134.09 MiB | 67.08 MiB/s 2024-06-26T07:21:17.9334041Z Receiving objects: 33% (9962/30187), 134.09 MiB | 67.08 MiB/s 2024-06-26T07:21:18.0219249Z Receiving objects: 34% (10264/30187), 170.35 MiB | 68.17 MiB/s 2024-06-26T07:21:18.0361210Z Receiving objects: 35% (10566/30187), 170.35 MiB | 68.17 MiB/s 2024-06-26T07:21:18.2495397Z Receiving objects: 36% (10868/30187), 170.35 MiB | 68.17 MiB/s 2024-06-26T07:21:18.2655749Z Receiving objects: 36% (11048/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.5871421Z Receiving objects: 37% (11170/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6109061Z Receiving objects: 38% (11472/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6122972Z Receiving objects: 39% (11773/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6153262Z Receiving objects: 40% (12075/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6168848Z Receiving objects: 41% (12377/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6184847Z Receiving objects: 42% (12679/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6197473Z Receiving objects: 43% (12981/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6208758Z Receiving objects: 44% (13283/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6232620Z Receiving objects: 45% (13585/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6245270Z Receiving objects: 46% (13887/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6250439Z Receiving objects: 47% (14188/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6258266Z Receiving objects: 48% (14490/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6284511Z Receiving objects: 49% (14792/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6303497Z Receiving objects: 50% (15094/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6316379Z Receiving objects: 51% (15396/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6555224Z Receiving objects: 52% (15698/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6558244Z Receiving objects: 53% (16000/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6567370Z Receiving objects: 54% (16301/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6576048Z Receiving objects: 55% (16603/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6581058Z Receiving objects: 56% (16905/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6601888Z Receiving objects: 57% (17207/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6613198Z Receiving objects: 58% (17509/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6624778Z Receiving objects: 59% (17811/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6640867Z Receiving objects: 60% (18113/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6650616Z Receiving objects: 61% (18415/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6661954Z Receiving objects: 62% (18716/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6811509Z Receiving objects: 63% (19018/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6818451Z Receiving objects: 64% (19320/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6825892Z Receiving objects: 65% (19622/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6833234Z Receiving objects: 66% (19924/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6844867Z Receiving objects: 67% (20226/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.6852770Z Receiving objects: 68% (20528/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.7135311Z Receiving objects: 69% (20830/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.7611727Z Receiving objects: 70% (21131/30187), 195.42 MiB | 65.16 MiB/s 2024-06-26T07:21:18.7617275Z Receiving objects: 71% (21433/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:18.7629962Z Receiving objects: 72% (21735/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:18.7644845Z Receiving objects: 73% (22037/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:18.7660088Z Receiving objects: 74% (22339/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:18.7672225Z Receiving objects: 75% (22641/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:18.7682073Z Receiving objects: 76% (22943/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:18.7703807Z Receiving objects: 77% (23244/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:18.7711476Z Receiving objects: 78% (23546/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:18.7721724Z Receiving objects: 79% (23848/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:18.7748348Z Receiving objects: 80% (24150/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0010928Z Receiving objects: 81% (24452/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0268442Z Receiving objects: 82% (24754/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0306058Z Receiving objects: 83% (25056/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0316551Z Receiving objects: 84% (25358/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0329759Z Receiving objects: 85% (25659/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0364826Z Receiving objects: 86% (25961/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0399995Z Receiving objects: 87% (26263/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0549836Z Receiving objects: 88% (26565/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0555285Z Receiving objects: 89% (26867/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0564200Z Receiving objects: 90% (27169/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0570433Z Receiving objects: 91% (27471/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0579681Z Receiving objects: 92% (27773/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0586246Z Receiving objects: 93% (28074/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0722871Z Receiving objects: 94% (28376/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0739636Z Receiving objects: 95% (28678/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0752029Z Receiving objects: 96% (28980/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0773381Z Receiving objects: 97% (29282/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0791308Z Receiving objects: 98% (29584/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0839693Z Receiving objects: 99% (29886/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0841037Z remote: Total 30187 (delta 256), reused 486 (delta 216), pack-reused 29641 2024-06-26T07:21:19.0862223Z Receiving objects: 100% (30187/30187), 228.12 MiB | 65.19 MiB/s 2024-06-26T07:21:19.0863121Z Receiving objects: 100% (30187/30187), 245.25 MiB | 63.92 MiB/s, done. 2024-06-26T07:21:19.0903218Z Resolving deltas: 0% (0/16205) 2024-06-26T07:21:19.0913870Z Resolving deltas: 1% (163/16205) 2024-06-26T07:21:19.0932015Z Resolving deltas: 2% (327/16205) 2024-06-26T07:21:19.0954995Z Resolving deltas: 3% (507/16205) 2024-06-26T07:21:19.0976362Z Resolving deltas: 4% (663/16205) 2024-06-26T07:21:19.0990271Z Resolving deltas: 5% (819/16205) 2024-06-26T07:21:19.1001188Z Resolving deltas: 6% (974/16205) 2024-06-26T07:21:19.1014876Z Resolving deltas: 7% (1143/16205) 2024-06-26T07:21:19.1028738Z Resolving deltas: 8% (1303/16205) 2024-06-26T07:21:19.1040277Z Resolving deltas: 9% (1472/16205) 2024-06-26T07:21:19.1053622Z Resolving deltas: 10% (1645/16205) 2024-06-26T07:21:19.1068106Z Resolving deltas: 11% (1783/16205) 2024-06-26T07:21:19.1078921Z Resolving deltas: 12% (1950/16205) 2024-06-26T07:21:19.1095379Z Resolving deltas: 13% (2110/16205) 2024-06-26T07:21:19.1106542Z Resolving deltas: 14% (2270/16205) 2024-06-26T07:21:19.1130537Z Resolving deltas: 15% (2431/16205) 2024-06-26T07:21:19.1151108Z Resolving deltas: 16% (2618/16205) 2024-06-26T07:21:19.1170967Z Resolving deltas: 17% (2761/16205) 2024-06-26T07:21:19.1173672Z Resolving deltas: 18% (3043/16205) 2024-06-26T07:21:19.1183625Z Resolving deltas: 19% (3087/16205) 2024-06-26T07:21:19.1212337Z Resolving deltas: 20% (3265/16205) 2024-06-26T07:21:19.1248470Z Resolving deltas: 21% (3417/16205) 2024-06-26T07:21:19.1277388Z Resolving deltas: 22% (3566/16205) 2024-06-26T07:21:19.1278339Z Resolving deltas: 23% (3860/16205) 2024-06-26T07:21:19.1291628Z Resolving deltas: 24% (3890/16205) 2024-06-26T07:21:19.1311455Z Resolving deltas: 25% (4056/16205) 2024-06-26T07:21:19.1314481Z Resolving deltas: 26% (4335/16205) 2024-06-26T07:21:19.1328458Z Resolving deltas: 27% (4383/16205) 2024-06-26T07:21:19.1377781Z Resolving deltas: 28% (4542/16205) 2024-06-26T07:21:19.1403450Z Resolving deltas: 29% (4701/16205) 2024-06-26T07:21:19.1420284Z Resolving deltas: 30% (4863/16205) 2024-06-26T07:21:19.1435874Z Resolving deltas: 31% (5030/16205) 2024-06-26T07:21:19.1457384Z Resolving deltas: 32% (5186/16205) 2024-06-26T07:21:19.1477198Z Resolving deltas: 33% (5348/16205) 2024-06-26T07:21:19.1489164Z Resolving deltas: 34% (5533/16205) 2024-06-26T07:21:19.1501167Z Resolving deltas: 35% (5703/16205) 2024-06-26T07:21:19.1515561Z Resolving deltas: 36% (5834/16205) 2024-06-26T07:21:19.1542908Z Resolving deltas: 37% (5998/16205) 2024-06-26T07:21:19.1560612Z Resolving deltas: 38% (6172/16205) 2024-06-26T07:21:19.1587673Z Resolving deltas: 39% (6323/16205) 2024-06-26T07:21:19.1593680Z Resolving deltas: 40% (6547/16205) 2024-06-26T07:21:19.1621235Z Resolving deltas: 41% (6660/16205) 2024-06-26T07:21:19.1635518Z Resolving deltas: 42% (6823/16205) 2024-06-26T07:21:19.1644976Z Resolving deltas: 43% (6998/16205) 2024-06-26T07:21:19.1672743Z Resolving deltas: 44% (7163/16205) 2024-06-26T07:21:19.1678905Z Resolving deltas: 45% (7363/16205) 2024-06-26T07:21:19.1688823Z Resolving deltas: 46% (7486/16205) 2024-06-26T07:21:19.1701869Z Resolving deltas: 47% (7640/16205) 2024-06-26T07:21:19.1728334Z Resolving deltas: 48% (7779/16205) 2024-06-26T07:21:19.1752433Z Resolving deltas: 49% (7942/16205) 2024-06-26T07:21:19.1766227Z Resolving deltas: 50% (8109/16205) 2024-06-26T07:21:19.1796984Z Resolving deltas: 51% (8265/16205) 2024-06-26T07:21:19.1808120Z Resolving deltas: 52% (8450/16205) 2024-06-26T07:21:19.1816390Z Resolving deltas: 53% (8590/16205) 2024-06-26T07:21:19.1827007Z Resolving deltas: 54% (8755/16205) 2024-06-26T07:21:19.1841131Z Resolving deltas: 55% (8936/16205) 2024-06-26T07:21:19.1853054Z Resolving deltas: 56% (9078/16205) 2024-06-26T07:21:19.1871674Z Resolving deltas: 57% (9238/16205) 2024-06-26T07:21:19.1880808Z Resolving deltas: 58% (9400/16205) 2024-06-26T07:21:19.1891710Z Resolving deltas: 59% (9609/16205) 2024-06-26T07:21:19.1894856Z Resolving deltas: 60% (9865/16205) 2024-06-26T07:21:19.1903468Z Resolving deltas: 61% (9919/16205) 2024-06-26T07:21:19.1915773Z Resolving deltas: 62% (10088/16205) 2024-06-26T07:21:19.1924868Z Resolving deltas: 63% (10321/16205) 2024-06-26T07:21:19.1926360Z Resolving deltas: 64% (10494/16205) 2024-06-26T07:21:19.1935097Z Resolving deltas: 65% (10535/16205) 2024-06-26T07:21:19.1966156Z Resolving deltas: 66% (10696/16205) 2024-06-26T07:21:19.1972676Z Resolving deltas: 68% (11170/16205) 2024-06-26T07:21:19.1992773Z Resolving deltas: 69% (11289/16205) 2024-06-26T07:21:19.2003084Z Resolving deltas: 70% (11360/16205) 2024-06-26T07:21:19.2017613Z Resolving deltas: 71% (11516/16205) 2024-06-26T07:21:19.2028034Z Resolving deltas: 72% (11706/16205) 2024-06-26T07:21:19.2048452Z Resolving deltas: 73% (11840/16205) 2024-06-26T07:21:19.2060558Z Resolving deltas: 74% (11997/16205) 2024-06-26T07:21:19.2100809Z Resolving deltas: 75% (12158/16205) 2024-06-26T07:21:19.2107984Z Resolving deltas: 78% (12696/16205) 2024-06-26T07:21:19.2125397Z Resolving deltas: 79% (12828/16205) 2024-06-26T07:21:19.2143327Z Resolving deltas: 80% (12977/16205) 2024-06-26T07:21:19.2162279Z Resolving deltas: 81% (13134/16205) 2024-06-26T07:21:19.2175032Z Resolving deltas: 82% (13300/16205) 2024-06-26T07:21:19.2183242Z Resolving deltas: 83% (13451/16205) 2024-06-26T07:21:19.2197638Z Resolving deltas: 84% (13613/16205) 2024-06-26T07:21:19.2223661Z Resolving deltas: 85% (13775/16205) 2024-06-26T07:21:19.2235068Z Resolving deltas: 86% (13937/16205) 2024-06-26T07:21:19.2242930Z Resolving deltas: 87% (14099/16205) 2024-06-26T07:21:19.2251813Z Resolving deltas: 88% (14268/16205) 2024-06-26T07:21:19.2262224Z Resolving deltas: 89% (14438/16205) 2024-06-26T07:21:19.2274242Z Resolving deltas: 90% (14606/16205) 2024-06-26T07:21:19.2283866Z Resolving deltas: 91% (14765/16205) 2024-06-26T07:21:19.2297130Z Resolving deltas: 92% (14917/16205) 2024-06-26T07:21:19.2302607Z Resolving deltas: 93% (15132/16205) 2024-06-26T07:21:19.2312235Z Resolving deltas: 94% (15246/16205) 2024-06-26T07:21:19.2322274Z Resolving deltas: 95% (15402/16205) 2024-06-26T07:21:19.2331777Z Resolving deltas: 96% (15561/16205) 2024-06-26T07:21:19.2341111Z Resolving deltas: 97% (15719/16205) 2024-06-26T07:21:19.2351416Z Resolving deltas: 98% (15887/16205) 2024-06-26T07:21:19.2393609Z Resolving deltas: 99% (16044/16205) 2024-06-26T07:21:19.2394159Z Resolving deltas: 100% (16205/16205) 2024-06-26T07:21:19.2394683Z Resolving deltas: 100% (16205/16205), done. 2024-06-26T07:21:19.8878071Z + pushd torchbench 2024-06-26T07:21:19.8878839Z ~/workspace/torchbench ~/workspace 2024-06-26T07:21:19.8879797Z + git checkout 23512dbebd44a11eb84afbf53c3c071dd105297e 2024-06-26T07:21:19.9649302Z Note: switching to '23512dbebd44a11eb84afbf53c3c071dd105297e'. 2024-06-26T07:21:19.9650447Z 2024-06-26T07:21:19.9650997Z You are in 'detached HEAD' state. You can look around, make experimental 2024-06-26T07:21:19.9651969Z changes and commit them, and you can discard any commits you make in this 2024-06-26T07:21:19.9652926Z state without impacting any branches by switching back to a branch. 2024-06-26T07:21:19.9653492Z 2024-06-26T07:21:19.9653982Z If you want to create a new branch to retain commits you create, you may 2024-06-26T07:21:19.9655382Z do so (now or later) by using -c with the switch command. Example: 2024-06-26T07:21:19.9655910Z 2024-06-26T07:21:19.9656167Z git switch -c 2024-06-26T07:21:19.9656499Z 2024-06-26T07:21:19.9656650Z Or undo this operation with: 2024-06-26T07:21:19.9656958Z 2024-06-26T07:21:19.9657084Z git switch - 2024-06-26T07:21:19.9657289Z 2024-06-26T07:21:19.9657713Z Turn off this advice by setting config variable advice.detachedHead to false 2024-06-26T07:21:19.9658320Z 2024-06-26T07:21:19.9658914Z HEAD is now at 23512dbe Pin numpy version 2024-06-26T07:21:19.9659477Z + '[' hf_Bert ']' 2024-06-26T07:21:19.9660336Z + python install.py --continue_on_fail models hf_Bert hf_Albert nanogpt timm_vision_transformer 2024-06-26T07:21:22.7684956Z checking packages torch, torchvision, torchaudio are installed...OK 2024-06-26T07:21:53.4095437Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Albert...OK 2024-06-26T07:22:01.6750827Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert...OK 2024-06-26T07:22:01.6753890Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nanogpt...SKIP - No install.py is found 2024-06-26T07:22:01.6984878Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer...OK 2024-06-26T07:22:02.1761790Z + popd 2024-06-26T07:22:02.1762212Z ~/workspace 2024-06-26T07:22:02.1771320Z ++ pwd 2024-06-26T07:22:02.1775315Z + PYTHONPATH=/var/lib/jenkins/workspace/torchbench 2024-06-26T07:22:02.1775982Z + test_inductor_torchbench_smoketest_perf 2024-06-26T07:22:02.1782032Z ++ pwd 2024-06-26T07:22:02.1785344Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:22:02.1786252Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:22:02.1808102Z + TORCHINDUCTOR_ABI_COMPATIBLE=1 2024-06-26T07:22:02.1808653Z + TORCHINDUCTOR_CPP_WRAPPER=1 2024-06-26T07:22:02.1810624Z + python benchmarks/dynamo/torchbench.py --device cuda --accuracy --bfloat16 --inference --inductor --only hf_T5 --output /var/lib/jenkins/workspace/test/test-reports/inductor_cpp_wrapper_inference.csv 2024-06-26T07:22:05.8361991Z 2024-06-26T07:22:06.3448519Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:22:06.3448958Z 2024-06-26T07:22:06.3449671Z config.json: 0% 0.00/1.21k [00:00 will be ignored 2024-06-26T07:24:01.8547765Z [rank0]:W0626 07:24:01.853000 140190299944128 torch/_dynamo/backends/distributed.py:88] [2/0_1] Some buckets were extended beyond their requested parameter capacities in order to ensure each subgraph has an output node, required for fx graph partitioning. This can be the case when a subgraph would have only contained nodes performing inplace mutation, and returning no logical outputs. This should not be a problem, unless it results in too few graph partitions for optimal DDP performance. 2024-06-26T07:24:01.8688891Z [rank0]:W0626 07:24:01.867000 140190299944128 torch/_dynamo/backends/distributed.py:105] [2/0_1] DDPOptimizer extended these buckets to ensure per-subgraph output nodes: 2024-06-26T07:24:01.8702308Z [rank0]:W0626 07:24:01.867000 140190299944128 torch/_dynamo/backends/distributed.py:105] [2/0_1] ┌─────────┬─────────────┬────────────────────────┐ 2024-06-26T07:24:01.8706494Z [rank0]:W0626 07:24:01.867000 140190299944128 torch/_dynamo/backends/distributed.py:105] [2/0_1] │ Index │ Extra Ops │ Extra Param Size (b) │ 2024-06-26T07:24:01.8710145Z [rank0]:W0626 07:24:01.867000 140190299944128 torch/_dynamo/backends/distributed.py:105] [2/0_1] ├─────────┼─────────────┼────────────────────────┤ 2024-06-26T07:24:01.8713272Z [rank0]:W0626 07:24:01.867000 140190299944128 torch/_dynamo/backends/distributed.py:105] [2/0_1] │ 0 │ 157 │ 44910720 │ 2024-06-26T07:24:01.8715333Z [rank0]:W0626 07:24:01.867000 140190299944128 torch/_dynamo/backends/distributed.py:105] [2/0_1] └─────────┴─────────────┴────────────────────────┘ 2024-06-26T07:25:35.9224367Z [rank0]:W0626 07:25:35.920000 140190299944128 torch/_inductor/utils.py:1234] [3/0_1] DeviceCopy in input program 2024-06-26T07:25:49.1899442Z [rank0]:W0626 07:25:49.188000 140190299944128 torch/_dynamo/variables/tensor.py:714] [5/0] Graph break from `Tensor.item()`, consider setting: 2024-06-26T07:25:49.1902022Z [rank0]:W0626 07:25:49.188000 140190299944128 torch/_dynamo/variables/tensor.py:714] [5/0] torch._dynamo.config.capture_scalar_outputs = True 2024-06-26T07:25:49.1904916Z [rank0]:W0626 07:25:49.188000 140190299944128 torch/_dynamo/variables/tensor.py:714] [5/0] or: 2024-06-26T07:25:49.1906364Z [rank0]:W0626 07:25:49.188000 140190299944128 torch/_dynamo/variables/tensor.py:714] [5/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-06-26T07:25:49.1908093Z [rank0]:W0626 07:25:49.188000 140190299944128 torch/_dynamo/variables/tensor.py:714] [5/0] to include these operations in the captured graph. 2024-06-26T07:25:49.1909542Z [rank0]:W0626 07:25:49.188000 140190299944128 torch/_dynamo/variables/tensor.py:714] [5/0] 2024-06-26T07:26:34.5286112Z pass 2024-06-26T07:26:35.5386850Z [rank0]:[W626 07:26:35.191210524 ProcessGroupNCCL.cpp:1161] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator()) 2024-06-26T07:26:37.3231966Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_cpp_wrapper_inference.csv --expected benchmarks/dynamo/ci_expected_accuracy//inductor_torchbench_inference.csv 2024-06-26T07:26:37.8015967Z hf_T5 PASS 2024-06-26T07:26:37.8019987Z llama PASS 2024-06-26T07:26:37.8026272Z moco PASS 2024-06-26T07:26:37.9035978Z ++ realpath benchmarks/dynamo/torchbench_models_list.txt 2024-06-26T07:26:37.9050940Z + python benchmarks/dynamo/torchbench.py --device cuda --performance --backend inductor --float16 --training --batch-size-file /var/lib/jenkins/workspace/benchmarks/dynamo/torchbench_models_list.txt --only hf_Bert --output /var/lib/jenkins/workspace/test/test-reports/inductor_training_smoketest.csv 2024-06-26T07:26:41.6156619Z batch size: 16 2024-06-26T07:26:41.6165926Z 2024-06-26T07:26:45.3152416Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:26:45.3153008Z loading model: 0it [00:03, ?it/s] 2024-06-26T07:26:45.3196452Z cuda train hf_Bert 2024-06-26T07:27:26.6272516Z W0626 07:27:26.624000 140320225199296 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:28:17.1798658Z 2024-06-26T07:28:17.3946828Z running benchmark: 0% 0/30 [00:00 will be ignored 2024-06-26T07:30:07.8766175Z 2024-06-26T07:30:08.3047229Z running benchmark: 0% 0/30 [00:00 will be ignored 2024-06-26T07:32:20.1797694Z pass 2024-06-26T07:32:22.5772711Z Performing warm-start run for AlbertForQuestionAnswering 2024-06-26T07:32:26.6987074Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:32:26.6990785Z warnings.warn( 2024-06-26T07:32:27.3548912Z 2024-06-26T07:32:31.0914622Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:32:31.0915394Z loading model: 0it [00:03, ?it/s] 2024-06-26T07:32:31.0915951Z cuda train AlbertForQuestionAnswering 2024-06-26T07:32:50.7917500Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:32:50.7919943Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 582, in torch_dynamo_resume_in_forward_and_backward_pass_at_580 2024-06-26T07:32:50.7921834Z pred = mod(**cloned_inputs) 2024-06-26T07:32:50.7924698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:32:50.7926033Z return forward_call(*args, **kwargs) 2024-06-26T07:32:50.7928699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/albert/modeling_albert.py", line 1256, in forward 2024-06-26T07:32:50.7929935Z outputs = self.albert( 2024-06-26T07:32:50.7931131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:32:50.7932201Z return forward_call(*args, **kwargs) 2024-06-26T07:32:50.7933585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/albert/modeling_albert.py", line 729, in forward 2024-06-26T07:32:50.7934798Z embedding_output = self.embeddings( 2024-06-26T07:32:50.7936080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:32:50.7937144Z return forward_call(*args, **kwargs) 2024-06-26T07:32:50.7938480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/albert/modeling_albert.py", line 250, in forward 2024-06-26T07:32:50.7939761Z inputs_embeds = self.word_embeddings(input_ids) 2024-06-26T07:32:50.7940195Z 2024-06-26T07:32:51.0380927Z W0626 07:32:51.036000 140571044709568 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:32:54.0652582Z pass 2024-06-26T07:32:56.1446555Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_AlbertForQuestionAnswering.csv --expected benchmarks/dynamo/ci_expected_accuracy//inductor_huggingface_training.csv 2024-06-26T07:32:56.6115040Z AlbertForQuestionAnswering PASS 2024-06-26T07:32:56.6705743Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-06-26T07:32:56.6708929Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only AllenaiLongformerBase --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_AllenaiLongformerBase.csv 2024-06-26T07:33:00.7578393Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:33:00.7582730Z warnings.warn( 2024-06-26T07:33:01.1748431Z Performing cold-start run for AllenaiLongformerBase 2024-06-26T07:33:05.1506407Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:33:05.1510892Z warnings.warn( 2024-06-26T07:33:05.8124268Z 2024-06-26T07:33:09.4344365Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:33:09.4344966Z loading model: 0it [00:03, ?it/s] 2024-06-26T07:33:09.4345510Z cuda train AllenaiLongformerBase 2024-06-26T07:33:12.4018536Z W0626 07:33:12.400000 140112699450560 torch/_dynamo/variables/tensor.py:714] [3/0] Graph break from `Tensor.item()`, consider setting: 2024-06-26T07:33:12.4020417Z W0626 07:33:12.400000 140112699450560 torch/_dynamo/variables/tensor.py:714] [3/0] torch._dynamo.config.capture_scalar_outputs = True 2024-06-26T07:33:12.4021799Z W0626 07:33:12.400000 140112699450560 torch/_dynamo/variables/tensor.py:714] [3/0] or: 2024-06-26T07:33:12.4023143Z W0626 07:33:12.400000 140112699450560 torch/_dynamo/variables/tensor.py:714] [3/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-06-26T07:33:12.4025382Z W0626 07:33:12.400000 140112699450560 torch/_dynamo/variables/tensor.py:714] [3/0] to include these operations in the captured graph. 2024-06-26T07:33:12.4026908Z W0626 07:33:12.400000 140112699450560 torch/_dynamo/variables/tensor.py:714] [3/0] 2024-06-26T07:34:54.5514901Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:34:54.5519493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1318, in torch_dynamo_resume_in_forward_at_1291 2024-06-26T07:34:54.5520950Z layer_outputs = layer_module( 2024-06-26T07:34:54.5522121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:34:54.5523189Z return forward_call(*args, **kwargs) 2024-06-26T07:34:54.5524703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1246, in forward 2024-06-26T07:34:54.5526287Z self_attn_outputs = self.attention( 2024-06-26T07:34:54.5527485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:34:54.5528563Z return forward_call(*args, **kwargs) 2024-06-26T07:34:54.5529953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1182, in forward 2024-06-26T07:34:54.5531167Z self_outputs = self.self( 2024-06-26T07:34:54.5532264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:34:54.5533321Z return forward_call(*args, **kwargs) 2024-06-26T07:34:54.5534745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 571, in forward 2024-06-26T07:34:54.5536060Z attn_scores = self._sliding_chunks_query_key_matmul( 2024-06-26T07:34:54.5537719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 836, in _sliding_chunks_query_key_matmul 2024-06-26T07:34:54.5539412Z query = self._chunk(query, window_overlap, getattr(self.config, "onnx_export", False)) 2024-06-26T07:34:54.5541083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 778, in _chunk 2024-06-26T07:34:54.5542500Z return hidden_states.as_strided(size=chunk_size, stride=chunk_stride) 2024-06-26T07:34:54.5543059Z 2024-06-26T07:34:55.1697797Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:34:55.1699838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1734, in forward 2024-06-26T07:34:55.1701174Z embedding_output = self.embeddings( 2024-06-26T07:34:55.1702524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:34:55.1703613Z return forward_call(*args, **kwargs) 2024-06-26T07:34:55.1705413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 470, in forward 2024-06-26T07:34:55.1706739Z inputs_embeds = self.word_embeddings(input_ids) 2024-06-26T07:34:55.1707166Z 2024-06-26T07:34:55.1791491Z W0626 07:34:55.178000 140112699450560 torch/_logging/_internal.py:1040] [16/0] Profiler function will be ignored 2024-06-26T07:35:50.9111753Z pass 2024-06-26T07:35:56.5041057Z Performing warm-start run for AllenaiLongformerBase 2024-06-26T07:36:00.5833685Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:36:00.5836817Z warnings.warn( 2024-06-26T07:36:01.2475492Z 2024-06-26T07:36:04.9206735Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:36:04.9207501Z loading model: 0it [00:03, ?it/s] 2024-06-26T07:36:04.9208048Z cuda train AllenaiLongformerBase 2024-06-26T07:36:07.8515614Z W0626 07:36:07.850000 140102911349952 torch/_dynamo/variables/tensor.py:714] [3/0] Graph break from `Tensor.item()`, consider setting: 2024-06-26T07:36:07.8517418Z W0626 07:36:07.850000 140102911349952 torch/_dynamo/variables/tensor.py:714] [3/0] torch._dynamo.config.capture_scalar_outputs = True 2024-06-26T07:36:07.8518803Z W0626 07:36:07.850000 140102911349952 torch/_dynamo/variables/tensor.py:714] [3/0] or: 2024-06-26T07:36:07.8520131Z W0626 07:36:07.850000 140102911349952 torch/_dynamo/variables/tensor.py:714] [3/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-06-26T07:36:07.8523830Z W0626 07:36:07.850000 140102911349952 torch/_dynamo/variables/tensor.py:714] [3/0] to include these operations in the captured graph. 2024-06-26T07:36:07.8525268Z W0626 07:36:07.850000 140102911349952 torch/_dynamo/variables/tensor.py:714] [3/0] 2024-06-26T07:36:50.7926474Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:36:50.7931863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1318, in torch_dynamo_resume_in_forward_at_1291 2024-06-26T07:36:50.7934505Z layer_outputs = layer_module( 2024-06-26T07:36:50.7936098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:36:50.7937171Z return forward_call(*args, **kwargs) 2024-06-26T07:36:50.7938615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1246, in forward 2024-06-26T07:36:50.7940074Z self_attn_outputs = self.attention( 2024-06-26T07:36:50.7941314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:36:50.7942401Z return forward_call(*args, **kwargs) 2024-06-26T07:36:50.7943786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1182, in forward 2024-06-26T07:36:50.7945395Z self_outputs = self.self( 2024-06-26T07:36:50.7946539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:36:50.7947594Z return forward_call(*args, **kwargs) 2024-06-26T07:36:50.7948991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 571, in forward 2024-06-26T07:36:50.7950311Z attn_scores = self._sliding_chunks_query_key_matmul( 2024-06-26T07:36:50.7952031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 836, in _sliding_chunks_query_key_matmul 2024-06-26T07:36:50.7953734Z query = self._chunk(query, window_overlap, getattr(self.config, "onnx_export", False)) 2024-06-26T07:36:50.7955466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 778, in _chunk 2024-06-26T07:36:50.7956920Z return hidden_states.as_strided(size=chunk_size, stride=chunk_stride) 2024-06-26T07:36:50.7957492Z 2024-06-26T07:36:51.0278851Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:36:51.0280828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1734, in forward 2024-06-26T07:36:51.0282122Z embedding_output = self.embeddings( 2024-06-26T07:36:51.0283322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:36:51.0284377Z return forward_call(*args, **kwargs) 2024-06-26T07:36:51.0285762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 470, in forward 2024-06-26T07:36:51.0287577Z inputs_embeds = self.word_embeddings(input_ids) 2024-06-26T07:36:51.0288007Z 2024-06-26T07:36:51.0367008Z W0626 07:36:51.035000 140102911349952 torch/_logging/_internal.py:1040] [16/0] Profiler function will be ignored 2024-06-26T07:37:11.4332493Z pass 2024-06-26T07:37:14.2009285Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_AllenaiLongformerBase.csv --expected benchmarks/dynamo/ci_expected_accuracy//inductor_huggingface_training.csv 2024-06-26T07:37:14.6725998Z AllenaiLongformerBase PASS 2024-06-26T07:37:14.7338010Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-06-26T07:37:14.7341242Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only DistilBertForMaskedLM --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_DistilBertForMaskedLM.csv 2024-06-26T07:37:18.8259549Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:37:18.8261857Z warnings.warn( 2024-06-26T07:37:19.2764924Z Performing cold-start run for DistilBertForMaskedLM 2024-06-26T07:37:23.2470512Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:37:23.2472846Z warnings.warn( 2024-06-26T07:37:23.7958801Z 2024-06-26T07:37:25.7833938Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:37:25.7834550Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:37:25.7835095Z cuda train DistilBertForMaskedLM 2024-06-26T07:37:53.1819584Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:37:53.1822892Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 582, in torch_dynamo_resume_in_forward_and_backward_pass_at_580 2024-06-26T07:37:53.1825422Z pred = mod(**cloned_inputs) 2024-06-26T07:37:53.1829463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:37:53.1830533Z return forward_call(*args, **kwargs) 2024-06-26T07:37:53.1832023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 905, in forward 2024-06-26T07:37:53.1833265Z dlbrt_output = self.distilbert( 2024-06-26T07:37:53.1834451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:37:53.1835521Z return forward_call(*args, **kwargs) 2024-06-26T07:37:53.1836919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2024-06-26T07:37:53.1838406Z embeddings = self.embeddings(input_ids, inputs_embeds) # (bs, seq_length, dim) 2024-06-26T07:37:53.1839902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:37:53.1840961Z return forward_call(*args, **kwargs) 2024-06-26T07:37:53.1842322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 141, in forward 2024-06-26T07:37:53.1843769Z input_embeds = self.word_embeddings(input_ids) # (bs, max_seq_length, dim) 2024-06-26T07:37:53.1844356Z 2024-06-26T07:37:53.3065619Z W0626 07:37:53.305000 140487956247744 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:38:21.6243659Z pass 2024-06-26T07:38:24.3660903Z Performing warm-start run for DistilBertForMaskedLM 2024-06-26T07:38:28.4407317Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:38:28.4411582Z warnings.warn( 2024-06-26T07:38:29.0662873Z 2024-06-26T07:38:31.0851435Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:38:31.0852263Z loading model: 0it [00:02, ?it/s] 2024-06-26T07:38:31.0852825Z cuda train DistilBertForMaskedLM 2024-06-26T07:38:43.3303710Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:38:43.3306486Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 582, in torch_dynamo_resume_in_forward_and_backward_pass_at_580 2024-06-26T07:38:43.3308794Z pred = mod(**cloned_inputs) 2024-06-26T07:38:43.3311307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:38:43.3312428Z return forward_call(*args, **kwargs) 2024-06-26T07:38:43.3316226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 905, in forward 2024-06-26T07:38:43.3317555Z dlbrt_output = self.distilbert( 2024-06-26T07:38:43.3318777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:38:43.3319833Z return forward_call(*args, **kwargs) 2024-06-26T07:38:43.3321234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2024-06-26T07:38:43.3322743Z embeddings = self.embeddings(input_ids, inputs_embeds) # (bs, seq_length, dim) 2024-06-26T07:38:43.3324217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:38:43.3325340Z return forward_call(*args, **kwargs) 2024-06-26T07:38:43.3326721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 141, in forward 2024-06-26T07:38:43.3328176Z input_embeds = self.word_embeddings(input_ids) # (bs, max_seq_length, dim) 2024-06-26T07:38:43.3328781Z 2024-06-26T07:38:43.4540656Z W0626 07:38:43.452000 140080591463616 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:38:53.2812193Z pass 2024-06-26T07:38:55.4639169Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_DistilBertForMaskedLM.csv --expected benchmarks/dynamo/ci_expected_accuracy//inductor_huggingface_training.csv 2024-06-26T07:38:55.9458400Z DistilBertForMaskedLM PASS 2024-06-26T07:38:56.0120558Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-06-26T07:38:56.0123593Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only DistillGPT2 --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_DistillGPT2.csv 2024-06-26T07:39:00.0670144Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:39:00.0672665Z warnings.warn( 2024-06-26T07:39:00.4422300Z Performing cold-start run for DistillGPT2 2024-06-26T07:39:04.4278833Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:39:04.4281497Z warnings.warn( 2024-06-26T07:39:05.0439384Z 2024-06-26T07:39:07.6092546Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:39:07.6093135Z loading model: 0it [00:02, ?it/s] 2024-06-26T07:39:07.6093663Z cuda train DistillGPT2 2024-06-26T07:39:33.5676823Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:39:33.5679275Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 582, in torch_dynamo_resume_in_forward_and_backward_pass_at_580 2024-06-26T07:39:33.5681350Z pred = mod(**cloned_inputs) 2024-06-26T07:39:33.5683669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:39:33.5685364Z return forward_call(*args, **kwargs) 2024-06-26T07:39:33.5686761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1074, in forward 2024-06-26T07:39:33.5687943Z transformer_outputs = self.transformer( 2024-06-26T07:39:33.5689165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:39:33.5690231Z return forward_call(*args, **kwargs) 2024-06-26T07:39:33.5691513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 837, in forward 2024-06-26T07:39:33.5692657Z inputs_embeds = self.wte(input_ids) 2024-06-26T07:39:33.5693016Z 2024-06-26T07:39:33.6737042Z W0626 07:39:33.672000 140333003633856 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:39:55.7863450Z pass 2024-06-26T07:39:58.3120668Z Performing warm-start run for DistillGPT2 2024-06-26T07:40:02.4510074Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:40:02.4512451Z warnings.warn( 2024-06-26T07:40:03.0525862Z 2024-06-26T07:40:05.6246531Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:40:05.6247106Z loading model: 0it [00:02, ?it/s] 2024-06-26T07:40:05.6247642Z cuda train DistillGPT2 2024-06-26T07:40:17.5689916Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:40:17.5691740Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 582, in torch_dynamo_resume_in_forward_and_backward_pass_at_580 2024-06-26T07:40:17.5693072Z pred = mod(**cloned_inputs) 2024-06-26T07:40:17.5695585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:40:17.5696669Z return forward_call(*args, **kwargs) 2024-06-26T07:40:17.5697986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1074, in forward 2024-06-26T07:40:17.5699252Z transformer_outputs = self.transformer( 2024-06-26T07:40:17.5700476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:40:17.5701702Z return forward_call(*args, **kwargs) 2024-06-26T07:40:17.5702997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 837, in forward 2024-06-26T07:40:17.5704472Z inputs_embeds = self.wte(input_ids) 2024-06-26T07:40:17.5704836Z 2024-06-26T07:40:17.6711998Z W0626 07:40:17.669000 140569520719040 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:40:25.0562369Z pass 2024-06-26T07:40:27.2797113Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_DistillGPT2.csv --expected benchmarks/dynamo/ci_expected_accuracy//inductor_huggingface_training.csv 2024-06-26T07:40:27.7551429Z DistillGPT2 PASS 2024-06-26T07:40:27.8197064Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-06-26T07:40:27.8200049Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only GoogleFnet --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_GoogleFnet.csv 2024-06-26T07:40:31.9095097Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:40:31.9099123Z warnings.warn( 2024-06-26T07:40:32.2731807Z Performing cold-start run for GoogleFnet 2024-06-26T07:40:36.3030537Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:40:36.3032710Z warnings.warn( 2024-06-26T07:40:36.8720427Z 2024-06-26T07:40:38.7765332Z loading model: 0it [00:00, ?it/s]WARNING:common:Model GoogleFnet supports float32 only 2024-06-26T07:40:39.1403168Z 2024-06-26T07:40:39.1404377Z loading model: 0it [00:02, ?it/s] 2024-06-26T07:40:39.1405566Z WARNING:common:Model GoogleFnet supports float32 only 2024-06-26T07:40:39.1420875Z cuda train GoogleFnet 2024-06-26T07:40:39.4115151Z WARNING:common:Model GoogleFnet supports float32 only 2024-06-26T07:41:05.1354452Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:41:05.1356759Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 582, in torch_dynamo_resume_in_forward_and_backward_pass_at_580 2024-06-26T07:41:05.1358025Z pred = mod(**cloned_inputs) 2024-06-26T07:41:05.1360526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:41:05.1361599Z return forward_call(*args, **kwargs) 2024-06-26T07:41:05.1362890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 749, in forward 2024-06-26T07:41:05.1364523Z outputs = self.fnet( 2024-06-26T07:41:05.1366046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:41:05.1367555Z return forward_call(*args, **kwargs) 2024-06-26T07:41:05.1369848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 581, in forward 2024-06-26T07:41:05.1371877Z embedding_output = self.embeddings( 2024-06-26T07:41:05.1373968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:41:05.1375714Z return forward_call(*args, **kwargs) 2024-06-26T07:41:05.1377419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 148, in forward 2024-06-26T07:41:05.1378641Z inputs_embeds = self.word_embeddings(input_ids) 2024-06-26T07:41:05.1379192Z 2024-06-26T07:41:05.2897753Z W0626 07:41:05.288000 140222640735424 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:41:34.0944675Z pass 2024-06-26T07:41:37.0141152Z Performing warm-start run for GoogleFnet 2024-06-26T07:41:41.1750896Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:41:41.1753800Z warnings.warn( 2024-06-26T07:41:41.7366390Z 2024-06-26T07:41:43.6521656Z loading model: 0it [00:00, ?it/s]WARNING:common:Model GoogleFnet supports float32 only 2024-06-26T07:41:43.9115438Z 2024-06-26T07:41:43.9116719Z loading model: 0it [00:02, ?it/s] 2024-06-26T07:41:43.9117830Z WARNING:common:Model GoogleFnet supports float32 only 2024-06-26T07:41:43.9131562Z cuda train GoogleFnet 2024-06-26T07:41:44.1408897Z WARNING:common:Model GoogleFnet supports float32 only 2024-06-26T07:41:55.6042757Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:41:55.6044842Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 582, in torch_dynamo_resume_in_forward_and_backward_pass_at_580 2024-06-26T07:41:55.6046154Z pred = mod(**cloned_inputs) 2024-06-26T07:41:55.6049622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:41:55.6051673Z return forward_call(*args, **kwargs) 2024-06-26T07:41:55.6053189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 749, in forward 2024-06-26T07:41:55.6054310Z outputs = self.fnet( 2024-06-26T07:41:55.6055407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:41:55.6056456Z return forward_call(*args, **kwargs) 2024-06-26T07:41:55.6057732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 581, in forward 2024-06-26T07:41:55.6058886Z embedding_output = self.embeddings( 2024-06-26T07:41:55.6060187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:41:55.6061252Z return forward_call(*args, **kwargs) 2024-06-26T07:41:55.6062540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 148, in forward 2024-06-26T07:41:55.6063728Z inputs_embeds = self.word_embeddings(input_ids) 2024-06-26T07:41:55.6064409Z 2024-06-26T07:41:55.7263233Z W0626 07:41:55.725000 140290389259456 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:42:05.5428976Z pass 2024-06-26T07:42:07.7195937Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_GoogleFnet.csv --expected benchmarks/dynamo/ci_expected_accuracy//inductor_huggingface_training.csv 2024-06-26T07:42:08.1957807Z GoogleFnet PASS 2024-06-26T07:42:08.2532401Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-06-26T07:42:08.2535487Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only YituTechConvBert --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_YituTechConvBert.csv 2024-06-26T07:42:12.3346948Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:42:12.3350946Z warnings.warn( 2024-06-26T07:42:12.7654350Z Performing cold-start run for YituTechConvBert 2024-06-26T07:42:16.7493114Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:42:16.7495866Z warnings.warn( 2024-06-26T07:42:17.2883880Z 2024-06-26T07:42:20.1841809Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:42:20.1842779Z loading model: 0it [00:02, ?it/s] 2024-06-26T07:42:20.1843758Z cuda train YituTechConvBert 2024-06-26T07:43:24.4624990Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:43:24.4627725Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 582, in torch_dynamo_resume_in_forward_and_backward_pass_at_580 2024-06-26T07:43:24.4629736Z pred = mod(**cloned_inputs) 2024-06-26T07:43:24.4633765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:43:24.4635670Z return forward_call(*args, **kwargs) 2024-06-26T07:43:24.4637273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 918, in forward 2024-06-26T07:43:24.4638527Z generator_hidden_states = self.convbert( 2024-06-26T07:43:24.4639791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:43:24.4640859Z return forward_call(*args, **kwargs) 2024-06-26T07:43:24.4642214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 834, in forward 2024-06-26T07:43:24.4643428Z hidden_states = self.embeddings( 2024-06-26T07:43:24.4644660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:43:24.4645733Z return forward_call(*args, **kwargs) 2024-06-26T07:43:24.4647127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 230, in forward 2024-06-26T07:43:24.4648379Z inputs_embeds = self.word_embeddings(input_ids) 2024-06-26T07:43:24.4648829Z 2024-06-26T07:43:24.7219910Z W0626 07:43:24.720000 139838333060288 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:44:39.3303785Z pass 2024-06-26T07:44:45.0233348Z Performing warm-start run for YituTechConvBert 2024-06-26T07:44:49.1978126Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-06-26T07:44:49.1980760Z warnings.warn( 2024-06-26T07:44:49.7628050Z 2024-06-26T07:44:52.6591953Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:44:52.6592560Z loading model: 0it [00:02, ?it/s] 2024-06-26T07:44:52.6593080Z cuda train YituTechConvBert 2024-06-26T07:45:21.6950690Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:45:21.6953655Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 582, in torch_dynamo_resume_in_forward_and_backward_pass_at_580 2024-06-26T07:45:21.6955928Z pred = mod(**cloned_inputs) 2024-06-26T07:45:21.6959732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:45:21.6961661Z return forward_call(*args, **kwargs) 2024-06-26T07:45:21.6964123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 918, in forward 2024-06-26T07:45:21.6966256Z generator_hidden_states = self.convbert( 2024-06-26T07:45:21.6968424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:45:21.6970350Z return forward_call(*args, **kwargs) 2024-06-26T07:45:21.6972516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 834, in forward 2024-06-26T07:45:21.6974120Z hidden_states = self.embeddings( 2024-06-26T07:45:21.6975976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:45:21.6977049Z return forward_call(*args, **kwargs) 2024-06-26T07:45:21.6978602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 230, in forward 2024-06-26T07:45:21.6980076Z inputs_embeds = self.word_embeddings(input_ids) 2024-06-26T07:45:21.6980506Z 2024-06-26T07:45:21.9497225Z W0626 07:45:21.948000 140083297469632 torch/_logging/_internal.py:1040] [4/0] Profiler function will be ignored 2024-06-26T07:45:48.1807383Z pass 2024-06-26T07:45:50.8063561Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_YituTechConvBert.csv --expected benchmarks/dynamo/ci_expected_accuracy//inductor_huggingface_training.csv 2024-06-26T07:45:51.2844432Z YituTechConvBert PASS 2024-06-26T07:45:51.3455031Z + cleanup_workspace 2024-06-26T07:45:51.3456981Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-06-26T07:45:51.3459861Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-06-26T07:45:51.3461897Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-06-26T07:45:51.3462693Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-06-26T07:45:51.3464407Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-06-26T07:45:51.3465556Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-06-26T07:45:51.3492403Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-06-26T07:45:51.9771226Z ##[group]Run cat test/**/*_toprint.log || true 2024-06-26T07:45:51.9771855Z cat test/**/*_toprint.log || true 2024-06-26T07:45:51.9785905Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:45:51.9786543Z env: 2024-06-26T07:45:51.9786868Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:51.9787398Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:51.9788335Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:51.9789155Z ##[endgroup] 2024-06-26T07:45:51.9908811Z cat: 'test/**/*_toprint.log': No such file or directory 2024-06-26T07:45:51.9947339Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-06-26T07:45:51.9947910Z kill "$MONITOR_SCRIPT_PID" 2024-06-26T07:45:51.9960909Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:45:51.9961525Z env: 2024-06-26T07:45:51.9961834Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:51.9962375Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:51.9963305Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:51.9964143Z MONITOR_SCRIPT_PID: 3596928 2024-06-26T07:45:51.9964580Z ##[endgroup] 2024-06-26T07:45:52.0232128Z Prepare all required actions 2024-06-26T07:45:52.0233301Z Getting action download info 2024-06-26T07:45:52.3235423Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-06-26T07:45:52.7439842Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-06-26T07:45:52.7440531Z with: 2024-06-26T07:45:52.7441183Z file-suffix: test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958 2024-06-26T07:45:52.7442049Z use-gha: anything-non-empty-to-use-gha 2024-06-26T07:45:52.7442584Z s3-bucket: gha-artifacts 2024-06-26T07:45:52.7442981Z env: 2024-06-26T07:45:52.7443293Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:52.7444155Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:52.7445327Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:52.7446150Z ##[endgroup] 2024-06-26T07:45:52.7568774Z ##[group]Run actions/upload-artifact@v3 2024-06-26T07:45:52.7569584Z with: 2024-06-26T07:45:52.7570474Z name: test-jsons-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958.zip 2024-06-26T07:45:52.7571573Z retention-days: 14 2024-06-26T07:45:52.7571967Z if-no-files-found: warn 2024-06-26T07:45:52.7572379Z path: test/**/*.json 2024-06-26T07:45:52.7572730Z env: 2024-06-26T07:45:52.7573036Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:52.7573575Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:52.7574498Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:52.7575292Z ##[endgroup] 2024-06-26T07:45:53.2819406Z With the provided path, there will be 4 files uploaded 2024-06-26T07:45:53.2822405Z Starting artifact upload 2024-06-26T07:45:53.2824568Z For more detailed logs during the artifact upload process, enable step-debugging: https://docs.github.com/actions/monitoring-and-troubleshooting-workflows/enabling-debug-logging#enabling-step-debug-logging 2024-06-26T07:45:53.2826356Z Artifact name is valid! 2024-06-26T07:45:53.4501502Z Container for artifact "test-jsons-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958.zip" successfully created. Starting upload of file(s) 2024-06-26T07:45:53.9952550Z Total size of all the files uploaded is 35485 bytes 2024-06-26T07:45:53.9953395Z File upload process has finished. Finalizing the artifact upload 2024-06-26T07:45:54.1181557Z Artifact has been finalized. All files have been successfully uploaded! 2024-06-26T07:45:54.1182219Z 2024-06-26T07:45:54.1182621Z The raw size of all the files that were specified for upload is 331049 bytes 2024-06-26T07:45:54.1184394Z The size of all the files that were uploaded is 35485 bytes. This takes into account any gzip compression used to reduce the upload size, time and storage 2024-06-26T07:45:54.1185522Z 2024-06-26T07:45:54.1187175Z Note: The size of downloaded zips can differ significantly from the reported size. For more information see: https://github.com/actions/upload-artifact#zipped-artifact-downloads 2024-06-26T07:45:54.1188508Z 2024-06-26T07:45:54.1189621Z Artifact test-jsons-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958.zip has been successfully uploaded! 2024-06-26T07:45:54.1308465Z ##[group]Run actions/upload-artifact@v3 2024-06-26T07:45:54.1308982Z with: 2024-06-26T07:45:54.1309885Z name: test-reports-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958.zip 2024-06-26T07:45:54.1310896Z retention-days: 14 2024-06-26T07:45:54.1311277Z if-no-files-found: ignore 2024-06-26T07:45:54.1311734Z path: test/**/*.xml test/**/*.csv 2024-06-26T07:45:54.1312191Z env: 2024-06-26T07:45:54.1312507Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:54.1313035Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:54.1313969Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:54.1314799Z ##[endgroup] 2024-06-26T07:45:54.6502524Z With the provided path, there will be 15 files uploaded 2024-06-26T07:45:54.6504534Z Starting artifact upload 2024-06-26T07:45:54.6506868Z For more detailed logs during the artifact upload process, enable step-debugging: https://docs.github.com/actions/monitoring-and-troubleshooting-workflows/enabling-debug-logging#enabling-step-debug-logging 2024-06-26T07:45:54.6508617Z Artifact name is valid! 2024-06-26T07:45:54.8367040Z Container for artifact "test-reports-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958.zip" successfully created. Starting upload of file(s) 2024-06-26T07:45:56.2635831Z Total size of all the files uploaded is 3161 bytes 2024-06-26T07:45:56.2636693Z File upload process has finished. Finalizing the artifact upload 2024-06-26T07:45:56.4099657Z Artifact has been finalized. All files have been successfully uploaded! 2024-06-26T07:45:56.4100285Z 2024-06-26T07:45:56.4100711Z The raw size of all the files that were specified for upload is 4567 bytes 2024-06-26T07:45:56.4102835Z The size of all the files that were uploaded is 3161 bytes. This takes into account any gzip compression used to reduce the upload size, time and storage 2024-06-26T07:45:56.4103953Z 2024-06-26T07:45:56.4105542Z Note: The size of downloaded zips can differ significantly from the reported size. For more information see: https://github.com/actions/upload-artifact#zipped-artifact-downloads 2024-06-26T07:45:56.4106853Z 2024-06-26T07:45:56.4107912Z Artifact test-reports-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958.zip has been successfully uploaded! 2024-06-26T07:45:56.4223773Z ##[group]Run actions/upload-artifact@v3 2024-06-26T07:45:56.4224665Z with: 2024-06-26T07:45:56.4225428Z name: logs-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958.zip 2024-06-26T07:45:56.4226353Z retention-days: 14 2024-06-26T07:45:56.4226732Z if-no-files-found: ignore 2024-06-26T07:45:56.4227200Z path: usage_log.txt test/**/*.log 2024-06-26T07:45:56.4227675Z env: 2024-06-26T07:45:56.4227970Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:56.4228505Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:56.4229422Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:56.4230225Z ##[endgroup] 2024-06-26T07:45:56.9551646Z Multiple search paths detected. Calculating the least common ancestor of all paths 2024-06-26T07:45:56.9553963Z The least common ancestor is /home/weiwangmeta/actions-runner/_work/pytorch/pytorch. This will be the root directory of the artifact 2024-06-26T07:45:56.9555362Z With the provided path, there will be 1 file uploaded 2024-06-26T07:45:56.9556831Z Starting artifact upload 2024-06-26T07:45:56.9558725Z For more detailed logs during the artifact upload process, enable step-debugging: https://docs.github.com/actions/monitoring-and-troubleshooting-workflows/enabling-debug-logging#enabling-step-debug-logging 2024-06-26T07:45:56.9560503Z Artifact name is valid! 2024-06-26T07:45:57.1352227Z Container for artifact "logs-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958.zip" successfully created. Starting upload of file(s) 2024-06-26T07:45:57.3744211Z Total size of all the files uploaded is 14610 bytes 2024-06-26T07:45:57.3745026Z File upload process has finished. Finalizing the artifact upload 2024-06-26T07:45:57.5251968Z Artifact has been finalized. All files have been successfully uploaded! 2024-06-26T07:45:57.5252606Z 2024-06-26T07:45:57.5253009Z The raw size of all the files that were specified for upload is 179256 bytes 2024-06-26T07:45:57.5254630Z The size of all the files that were uploaded is 14610 bytes. This takes into account any gzip compression used to reduce the upload size, time and storage 2024-06-26T07:45:57.5255749Z 2024-06-26T07:45:57.5257564Z Note: The size of downloaded zips can differ significantly from the reported size. For more information see: https://github.com/actions/upload-artifact#zipped-artifact-downloads 2024-06-26T07:45:57.5258903Z 2024-06-26T07:45:57.5259896Z Artifact logs-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_26688887958.zip has been successfully uploaded! 2024-06-26T07:45:57.5363553Z ##[group]Run # shellcheck disable=SC2156 2024-06-26T07:45:57.5364135Z # shellcheck disable=SC2156 2024-06-26T07:45:57.5365128Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-06-26T07:45:57.5378377Z shell: /usr/bin/bash -e {0} 2024-06-26T07:45:57.5378785Z env: 2024-06-26T07:45:57.5379098Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:57.5379690Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:57.5380622Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:57.5381415Z ##[endgroup] 2024-06-26T07:45:57.8470743Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-06-26T07:45:57.8471634Z with: 2024-06-26T07:45:57.8471932Z env: 2024-06-26T07:45:57.8472245Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:57.8472770Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:57.8473698Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:57.8474583Z ##[endgroup] 2024-06-26T07:45:57.8497389Z ##[group]Run set -eou pipefail 2024-06-26T07:45:57.8497885Z set -eou pipefail 2024-06-26T07:45:57.8498294Z  2024-06-26T07:45:57.8498911Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-06-26T07:45:57.8499684Z for _ in $(seq 1440); do 2024-06-26T07:45:57.8500247Z  # Break if no ssh session exists anymore 2024-06-26T07:45:57.8500853Z  if [ "$(who)" = "" ]; then 2024-06-26T07:45:57.8501341Z  break 2024-06-26T07:45:57.8501928Z  fi 2024-06-26T07:45:57.8502273Z  echo "." 2024-06-26T07:45:57.8502668Z  sleep 5 2024-06-26T07:45:57.8503036Z done 2024-06-26T07:45:57.8516445Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:45:57.8517071Z env: 2024-06-26T07:45:57.8517379Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:57.8517929Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:57.8518859Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:57.8519659Z ##[endgroup] 2024-06-26T07:45:57.8555809Z Holding runner for 2 hours until all ssh sessions have logged out 2024-06-26T07:45:57.8616171Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:45:57.8617188Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:45:57.8617916Z # shellcheck disable=SC2046 2024-06-26T07:45:57.8618488Z docker stop $(docker ps -q) || true 2024-06-26T07:45:57.8619068Z # Prune all of the docker images 2024-06-26T07:45:57.8619643Z docker system prune -af 2024-06-26T07:45:57.8634776Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:45:57.8635396Z env: 2024-06-26T07:45:57.8635715Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:45:57.8636267Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:45:57.8637185Z DOCKER_CONTAINER_ID: 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:57.8637995Z ##[endgroup] 2024-06-26T07:45:58.2815423Z 15cecb048bf7 2024-06-26T07:45:59.1898228Z Deleted Containers: 2024-06-26T07:45:59.1899238Z 15cecb048bf7649c72f850363ee67df11514142aba3f7ea3064fc3da905b7710 2024-06-26T07:45:59.1900059Z 2024-06-26T07:46:06.4976203Z Deleted Images: 2024-06-26T07:46:06.4979383Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:46:06.4982400Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb 2024-06-26T07:46:06.4984942Z deleted: sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd 2024-06-26T07:46:06.4986676Z deleted: sha256:754ead7012f0b7691885a9660f02e80be6cee8c5665486fb995c1fb06adb6e92 2024-06-26T07:46:06.4988343Z deleted: sha256:fbe4496d8830e11be86d8f222522e7956f32f85bfc06b05ea53961f34a58af74 2024-06-26T07:46:06.4989692Z deleted: sha256:1c83814aa1a1a7e0cc998e3479bdab8e8766b975560cbf00690acf39ed899a80 2024-06-26T07:46:06.4990772Z deleted: sha256:e9ea8e61942c7f8ae4aab97b8d102b01d6bb4b6cf2a605808cd34130a17d8a0f 2024-06-26T07:46:06.4993253Z deleted: sha256:acc5fc32588cd95e082a6ce35e48d0f303ae8f96e3ab65c4e375b85e1f3c4a92 2024-06-26T07:46:06.4994333Z deleted: sha256:a93e53400904d29e4185d3be942ea34f19572ea438962788ae4e9b3d9bf4c498 2024-06-26T07:46:06.4995741Z deleted: sha256:c97092b88b5e4c77c68bbc4b51afab64e38719af1ff58013ec49a696cdd40a6e 2024-06-26T07:46:06.4996985Z deleted: sha256:b132889c90d67ffa53655846603e3fcc2372c23f04dbdbc54690a39d304eb89b 2024-06-26T07:46:06.4998027Z deleted: sha256:915d54081ed607710624abe3f2490114c745858831f4fc13396bd3b4b9a121bc 2024-06-26T07:46:06.4999152Z deleted: sha256:bfa8d446c498c87f535aef102695564de1b22c755e6c63944ae2f5073a2b31ec 2024-06-26T07:46:06.5000219Z deleted: sha256:e71460fb363c29e70bbe357d680aaa7ae694f094ef2f2713609158ad90611744 2024-06-26T07:46:06.5001278Z deleted: sha256:84079bb1b4ee1d0119ae3d3e701de460a1324912297c675425c9a36d30fcdb0f 2024-06-26T07:46:06.5002327Z deleted: sha256:d86ac3ca148d4758cb03b493fdd49109d8f1a2c7c85c4b755493c3c4529553e5 2024-06-26T07:46:06.5003387Z deleted: sha256:532b3ab97c44a673a0d5f62ee46fa6a83248a84de8877754a612355be56ed0a3 2024-06-26T07:46:06.5004461Z deleted: sha256:617e2a35f25ae24500c88c0c9b8475f0687fbcbed3d200013b4c99c6d8941aff 2024-06-26T07:46:06.5005534Z deleted: sha256:b88120bcd5a26a660c09c5df13176e0ef098fb2c8cdb57d74ab0796a3d1194d6 2024-06-26T07:46:06.5006585Z deleted: sha256:c016b2e7764735243205fb76fa9a273bb3dd4bf05532288e3a613ce022131ac3 2024-06-26T07:46:06.5007662Z deleted: sha256:bcaca462c390deddd2995de2d245a7495e7d2e1e94f0edad30a573c83d0588fd 2024-06-26T07:46:06.5008745Z deleted: sha256:9e260eba003d5c25d7b410e5c4c3093845d9e46611896840901c689d7fbf8ee6 2024-06-26T07:46:06.5009825Z deleted: sha256:3e62b86f8b911a9b59c62d55f063e71683ae6a01c22418db5d27930712adaad6 2024-06-26T07:46:06.5010857Z deleted: sha256:905920ccb1d88449b8dde686c34a4b2a8b6f59e78ac44916b189ea3593b9fbbb 2024-06-26T07:46:06.5011897Z deleted: sha256:961201237e03acd5061406238dd7a3c1343ac888d7583212cd0ab5ae1b4f0540 2024-06-26T07:46:06.5013129Z deleted: sha256:70047a2aac62159fa28fb4dbd4567ec2ffbab68fe046967e09acb0eb3e5cb9cb 2024-06-26T07:46:06.5014285Z deleted: sha256:1ec680ad781436ae65fa987f5fa1ed032ecd5c82a6e35eea6707de501f62c0f4 2024-06-26T07:46:06.5015386Z deleted: sha256:ef859623a50e85abf6e6c1dbd6188aa765a66cc7b2bd618a1cbe4825b7b2bd91 2024-06-26T07:46:06.5016463Z deleted: sha256:404ade5cacff59773b93ab438bfa64cd3a171d07f6f168bdc4a5d4bfa0b43526 2024-06-26T07:46:06.5017563Z deleted: sha256:5b27de6ee3d9f2f18217bf18dec725f7b4954c0e48c85d35752634c22d62209b 2024-06-26T07:46:06.5018644Z deleted: sha256:498e39c575b3fa069acaac7e46b91b4e54ae3a05f2ada5c4347091f346f25d4c 2024-06-26T07:46:06.5019712Z deleted: sha256:1405919d2694d65669b1f4c842282021afe89208ce86570e96464903733da2d0 2024-06-26T07:46:06.5020729Z deleted: sha256:d5b41e5d70896c0472fc3e6da0516a17d89cd6635dc75010ecddc31ce9690fad 2024-06-26T07:46:06.5021810Z deleted: sha256:18e9379b62b9d0bb40293d28a652eafaee66a499f66f0776bf14e37a513101b7 2024-06-26T07:46:06.5022883Z deleted: sha256:a781f80ed00ff2926a699fd01a309bb0b4fb7d5655eb652c205bb9c7f811035f 2024-06-26T07:46:06.5023942Z deleted: sha256:c0430973b7cf759075330ee29c79bfd1312c91e6033d9dd883b4b1c29743a171 2024-06-26T07:46:06.5025286Z deleted: sha256:e733b3d66efb9342e5a283376c0e1ddffa47a42fecc26e61ecaf0a3472885d97 2024-06-26T07:46:06.5026367Z deleted: sha256:8929c0b2047f74b84c82c39eff9077a92ae1a828cde00c9ab78fb3e0ed5e0283 2024-06-26T07:46:06.5027445Z deleted: sha256:f5b56155fb58a0a60fe6391d3581cf91610ed69d4f0a321257c19944da296a52 2024-06-26T07:46:06.5028499Z deleted: sha256:b54d2cb355238d1aa91e58df93064447b80f53706d1ca62823acc834f5ac54fe 2024-06-26T07:46:06.5029587Z deleted: sha256:e7311285af9eeddb51aa5629aa98386e89f1456ff32843b6aee6443247edf66c 2024-06-26T07:46:06.5030651Z deleted: sha256:67657ceec347da8dc8085d5731633efeac115ba677fa4b7c5966c4260f7abab0 2024-06-26T07:46:06.5031721Z deleted: sha256:b496e50409dbf414930fc9b4437aa0aa4ce27bc7840e050ae459aaffcf9aca86 2024-06-26T07:46:06.5032787Z deleted: sha256:8f5276170cc4a60d36ad7b964a502306b29546254cedcbcb101ea9123497d2c0 2024-06-26T07:46:06.5033827Z deleted: sha256:53d524d1564745e96a7e515462171b73184d9f1ff5454b32fcfbbca39d361467 2024-06-26T07:46:06.5034883Z deleted: sha256:2acbadb00d69b6aca575f04baba4ea26ae2b5c2d85f09945004ad93a78a8eb75 2024-06-26T07:46:06.5036219Z deleted: sha256:02665efeff878aeb603c1edb2e21cb700c171c025d64f5aae1b2b20b12f3964a 2024-06-26T07:46:06.5037444Z deleted: sha256:19bd23e9a51dfe883a8b8d2ff6d66ff1b3476b8912856406bb79b81762aa6d4b 2024-06-26T07:46:06.5038514Z deleted: sha256:73aa8637bf9299d6298dabd858480c274a62117388f2ca1d5e9eb15c6d1e4155 2024-06-26T07:46:06.5039586Z deleted: sha256:5f8968406305f348b1caebc0e035187d6e9965ace7b9d5f2f21114d37c9fe2e1 2024-06-26T07:46:06.5040644Z deleted: sha256:d827104914db0c503cf5c81f13e2456bf4efce42d4cf1f8d5d70838207278bee 2024-06-26T07:46:06.5041706Z deleted: sha256:867b219724f2f1aaf56f42e3e9f03a5d64bdca8a6a72407efd10328aeea5d510 2024-06-26T07:46:06.5042787Z deleted: sha256:b9ff2a7cceb9237b26bcb17728edf8a558eec5a76f63626e84f80541dfc0f1d4 2024-06-26T07:46:06.5043852Z deleted: sha256:be73f509106819007ce2b0f5a1301ac56ae583cb6c1fb69bee93efa0840368f1 2024-06-26T07:46:06.5044919Z deleted: sha256:2a11c07c915c3fdbb4e73096dea73ea950c970d50fe1a5764060bd62b7b7cf2c 2024-06-26T07:46:06.5045983Z deleted: sha256:5e9989622e2abc1c72f8184343507b45d75e4ff971edb68b96cbcd75ae484ff5 2024-06-26T07:46:06.5047059Z deleted: sha256:05ce642bd9cf5a9205bff107df28b1d6cef7d5f6ab3ea944d28c032ec61270df 2024-06-26T07:46:06.5048106Z deleted: sha256:017702617f6c07b73f2088678f4710b794cfe897a213a5cf122a0e2519eae0ac 2024-06-26T07:46:06.5049184Z deleted: sha256:44b3b37bb1ca39dc5cf356de41a23a5c12f605a1992813e408087fb2b0ad6d1c 2024-06-26T07:46:06.5050251Z deleted: sha256:8c387982e9f27db357a1d149dc149cabc0bd04668ad6956387a5072682dada67 2024-06-26T07:46:06.5051317Z deleted: sha256:10d415dca4e3fbb88a747b40317f3c6333c041ba8f73855f67fb9fe981fae5ee 2024-06-26T07:46:06.5052369Z deleted: sha256:6c65469bf0f53dcd625faefe363b8f3136c08388a6cf64d9158e8576e4ee89b1 2024-06-26T07:46:06.5053454Z deleted: sha256:027db5d8b76deeefbd8c34b485ab7ff14ed2a3e6800425c1ce0d77c2e9fdd4ab 2024-06-26T07:46:06.5054523Z deleted: sha256:24941234ad28cf5c8174d7cf18326c10b4e17f2c2137672ea0322df461597dd7 2024-06-26T07:46:06.5055588Z deleted: sha256:240ea2eebfa8519d6c2d88f4c99e66ce4e038d000f93ce8d03726c01d81180ef 2024-06-26T07:46:06.5056661Z deleted: sha256:09e823cb391a69f6a195e3ca8a41624b5f8b34eee14c6eb36277de814a91966e 2024-06-26T07:46:06.5057692Z deleted: sha256:730479d695169a6593bf0ac16b377491dce65175a6d158e012422367d04577fd 2024-06-26T07:46:06.5058738Z deleted: sha256:b0a30f9f7bd5459ff662453b9346a7ba8ea8db1191528345c1953eab4aa2f54e 2024-06-26T07:46:06.5059839Z deleted: sha256:75cefa87effaa3ad361c74642839923101f0f9982f814f3c9e2ed4ad52f2a008 2024-06-26T07:46:06.5060887Z deleted: sha256:6f97db87ba9512e615e871c6982000602982e1e5a5be334a56e7292251e84e00 2024-06-26T07:46:06.5061928Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-06-26T07:46:06.5062990Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-06-26T07:46:06.5064233Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-06-26T07:46:06.5065354Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-06-26T07:46:06.5066424Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-06-26T07:46:06.5067487Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-06-26T07:46:06.5068575Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-06-26T07:46:06.5069708Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-06-26T07:46:06.5070785Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-06-26T07:46:06.5071874Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-06-26T07:46:06.5072934Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-06-26T07:46:06.5073571Z 2024-06-26T07:46:06.5073783Z Total reclaimed space: 32.46GB 2024-06-26T07:46:06.5213155Z Post job cleanup. 2024-06-26T07:46:06.5278928Z Post job cleanup. 2024-06-26T07:46:06.6345632Z [command]/usr/bin/git version 2024-06-26T07:46:06.6397655Z git version 2.25.1 2024-06-26T07:46:06.6448464Z Temporarily overriding HOME='/home/weiwangmeta/actions-runner/_work/_temp/aed21506-c3cc-48c4-8f82-f0b8e9583f87' before making global git config changes 2024-06-26T07:46:06.6450588Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:46:06.6455516Z [command]/usr/bin/git config --global --add safe.directory /home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-06-26T07:46:06.6504854Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:46:06.6550538Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:46:06.6856052Z Entering 'android/libs/fbjni' 2024-06-26T07:46:06.6907504Z Entering 'third_party/FP16' 2024-06-26T07:46:06.6956099Z Entering 'third_party/FXdiv' 2024-06-26T07:46:06.7003571Z Entering 'third_party/NNPACK' 2024-06-26T07:46:06.7052744Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:46:06.7100273Z Entering 'third_party/XNNPACK' 2024-06-26T07:46:06.7170054Z Entering 'third_party/benchmark' 2024-06-26T07:46:06.7219884Z Entering 'third_party/cpp-httplib' 2024-06-26T07:46:06.7267428Z Entering 'third_party/cpuinfo' 2024-06-26T07:46:06.7315653Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:46:06.7364098Z Entering 'third_party/cutlass' 2024-06-26T07:46:06.7423491Z Entering 'third_party/eigen' 2024-06-26T07:46:06.7474077Z Entering 'third_party/fbgemm' 2024-06-26T07:46:06.7522739Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:46:06.7568457Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:46:06.7614941Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:46:06.7670318Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:46:06.7717307Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:46:06.7765653Z Entering 'third_party/flatbuffers' 2024-06-26T07:46:06.7817149Z Entering 'third_party/fmt' 2024-06-26T07:46:06.7866043Z Entering 'third_party/foxi' 2024-06-26T07:46:06.7913904Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:46:06.7961288Z Entering 'third_party/gloo' 2024-06-26T07:46:06.8009480Z Entering 'third_party/googletest' 2024-06-26T07:46:06.8058424Z Entering 'third_party/ideep' 2024-06-26T07:46:06.8104760Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:46:06.8160582Z Entering 'third_party/ittapi' 2024-06-26T07:46:06.8208305Z Entering 'third_party/kineto' 2024-06-26T07:46:06.8255444Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:46:06.8301610Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:46:06.8350915Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:46:06.8397442Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:46:06.8445111Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:46:06.8490497Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:46:06.8539953Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:46:06.8585714Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:46:06.8632768Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:46:06.8680764Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:46:06.8731105Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:46:06.8777534Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:46:06.8825765Z Entering 'third_party/mimalloc' 2024-06-26T07:46:06.8876410Z Entering 'third_party/nccl/nccl' 2024-06-26T07:46:06.8923872Z Entering 'third_party/nlohmann' 2024-06-26T07:46:06.8974528Z Entering 'third_party/onnx' 2024-06-26T07:46:06.9042508Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:46:06.9091322Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:46:06.9141588Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:46:06.9191853Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:46:06.9238995Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:46:06.9284944Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:46:06.9331360Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:46:06.9379319Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:46:06.9425281Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:46:06.9471176Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:46:06.9516406Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:46:06.9565113Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:46:06.9615816Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:46:06.9687702Z Entering 'third_party/pocketfft' 2024-06-26T07:46:06.9736920Z Entering 'third_party/protobuf' 2024-06-26T07:46:06.9790270Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:46:06.9837746Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:46:06.9889591Z Entering 'third_party/psimd' 2024-06-26T07:46:06.9938906Z Entering 'third_party/pthreadpool' 2024-06-26T07:46:06.9988941Z Entering 'third_party/pybind11' 2024-06-26T07:46:07.0039196Z Entering 'third_party/python-peachpy' 2024-06-26T07:46:07.0087329Z Entering 'third_party/sleef' 2024-06-26T07:46:07.0137040Z Entering 'third_party/tensorpipe' 2024-06-26T07:46:07.0185330Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:46:07.0233074Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:46:07.0278964Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:46:07.0328141Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:46:07.0375572Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:46:07.0447154Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:46:07.0480940Z http.https://github.com/.extraheader 2024-06-26T07:46:07.0494152Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-06-26T07:46:07.0534967Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:46:07.0823600Z Entering 'android/libs/fbjni' 2024-06-26T07:46:07.0850141Z http.https://github.com/.extraheader 2024-06-26T07:46:07.0887985Z Entering 'third_party/FP16' 2024-06-26T07:46:07.0915553Z http.https://github.com/.extraheader 2024-06-26T07:46:07.0952924Z Entering 'third_party/FXdiv' 2024-06-26T07:46:07.0979655Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1018240Z Entering 'third_party/NNPACK' 2024-06-26T07:46:07.1045812Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1081460Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:46:07.1110456Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1146835Z Entering 'third_party/XNNPACK' 2024-06-26T07:46:07.1173458Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1231046Z Entering 'third_party/benchmark' 2024-06-26T07:46:07.1259405Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1296648Z Entering 'third_party/cpp-httplib' 2024-06-26T07:46:07.1324433Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1362099Z Entering 'third_party/cpuinfo' 2024-06-26T07:46:07.1389808Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1427328Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:46:07.1453438Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1491232Z Entering 'third_party/cutlass' 2024-06-26T07:46:07.1517778Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1564664Z Entering 'third_party/eigen' 2024-06-26T07:46:07.1591676Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1631358Z Entering 'third_party/fbgemm' 2024-06-26T07:46:07.1657640Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1694065Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:46:07.1721080Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1757971Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:46:07.1783341Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1820890Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:46:07.1847394Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1893401Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:46:07.1920805Z http.https://github.com/.extraheader 2024-06-26T07:46:07.1957908Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:46:07.1984899Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2023942Z Entering 'third_party/flatbuffers' 2024-06-26T07:46:07.2051801Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2092219Z Entering 'third_party/fmt' 2024-06-26T07:46:07.2120576Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2158743Z Entering 'third_party/foxi' 2024-06-26T07:46:07.2186339Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2222791Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:46:07.2249100Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2454487Z Entering 'third_party/gloo' 2024-06-26T07:46:07.2481222Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2518372Z Entering 'third_party/googletest' 2024-06-26T07:46:07.2545506Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2583374Z Entering 'third_party/ideep' 2024-06-26T07:46:07.2611019Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2647073Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:46:07.2674345Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2724379Z Entering 'third_party/ittapi' 2024-06-26T07:46:07.2751856Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2790531Z Entering 'third_party/kineto' 2024-06-26T07:46:07.2818458Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2856752Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:46:07.2883965Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2924330Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:46:07.2951582Z http.https://github.com/.extraheader 2024-06-26T07:46:07.2991969Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:46:07.3019766Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3058984Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:46:07.3087299Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3125242Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:46:07.3152919Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3190286Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:46:07.3216884Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3256703Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:46:07.3283480Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3323435Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:46:07.3350139Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3388303Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:46:07.3413989Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3452205Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:46:07.3479597Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3518934Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:46:07.3545030Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3582892Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:46:07.3609925Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3650167Z Entering 'third_party/mimalloc' 2024-06-26T07:46:07.3677860Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3714670Z Entering 'third_party/nccl/nccl' 2024-06-26T07:46:07.3741820Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3779476Z Entering 'third_party/nlohmann' 2024-06-26T07:46:07.3806016Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3844823Z Entering 'third_party/onnx' 2024-06-26T07:46:07.3872278Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3927455Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:46:07.3954068Z http.https://github.com/.extraheader 2024-06-26T07:46:07.3991095Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:46:07.4017804Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4057827Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:46:07.4084936Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4123724Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:46:07.4149268Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4187119Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:46:07.4212775Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4249465Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:46:07.4275153Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4313294Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:46:07.4341047Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4378798Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:46:07.4404150Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4441219Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:46:07.4466966Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4503833Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:46:07.4530852Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4566839Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:46:07.4593605Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4634446Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:46:07.4661749Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4703179Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:46:07.4732906Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4796480Z Entering 'third_party/pocketfft' 2024-06-26T07:46:07.4823777Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4861052Z Entering 'third_party/protobuf' 2024-06-26T07:46:07.4888851Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4930785Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:46:07.4957053Z http.https://github.com/.extraheader 2024-06-26T07:46:07.4993845Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:46:07.5020508Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5059525Z Entering 'third_party/psimd' 2024-06-26T07:46:07.5086480Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5123329Z Entering 'third_party/pthreadpool' 2024-06-26T07:46:07.5151057Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5188179Z Entering 'third_party/pybind11' 2024-06-26T07:46:07.5214619Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5251199Z Entering 'third_party/python-peachpy' 2024-06-26T07:46:07.5278423Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5315156Z Entering 'third_party/sleef' 2024-06-26T07:46:07.5343157Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5379865Z Entering 'third_party/tensorpipe' 2024-06-26T07:46:07.5406903Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5444029Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:46:07.5470944Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5507253Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:46:07.5533497Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5570281Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:46:07.5598260Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5635508Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:46:07.5662992Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5699424Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:46:07.5728442Z http.https://github.com/.extraheader 2024-06-26T07:46:07.5879179Z A job completed hook has been configured by the self-hosted runner administrator 2024-06-26T07:46:07.5905493Z ##[group]Run '/home/weiwangmeta/post-job.sh' 2024-06-26T07:46:07.5917678Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:46:07.5918328Z ##[endgroup] 2024-06-26T07:46:07.5955625Z The approximate cost for this shard is below in $ 2024-06-26T07:46:07.5986756Z 2.916 2024-06-26T07:46:07.6211223Z Cleaning up orphan processes