2024-08-21T01:24:28.9649275Z Current runner version: '2.319.1' 2024-08-21T01:24:28.9658196Z Runner name: 'gh-ci-gcp-a100-4' 2024-08-21T01:24:28.9659171Z Runner group name: 'Default' 2024-08-21T01:24:28.9660194Z Machine name: 'gh-ci-gcp-a100-4' 2024-08-21T01:24:28.9682981Z Testing runner upgrade compatibility 2024-08-21T01:24:29.0546145Z ##[group]GITHUB_TOKEN Permissions 2024-08-21T01:24:29.0549616Z Actions: read 2024-08-21T01:24:29.0550206Z Attestations: read 2024-08-21T01:24:29.0550741Z Checks: read 2024-08-21T01:24:29.0551369Z Contents: read 2024-08-21T01:24:29.0551914Z Deployments: read 2024-08-21T01:24:29.0552508Z Discussions: read 2024-08-21T01:24:29.0553000Z Issues: read 2024-08-21T01:24:29.0553505Z Metadata: read 2024-08-21T01:24:29.0554064Z Packages: read 2024-08-21T01:24:29.0554548Z Pages: read 2024-08-21T01:24:29.0555061Z PullRequests: read 2024-08-21T01:24:29.0555769Z RepositoryProjects: read 2024-08-21T01:24:29.0556416Z SecurityEvents: read 2024-08-21T01:24:29.0556967Z Statuses: read 2024-08-21T01:24:29.0557771Z ##[endgroup] 2024-08-21T01:24:29.0562103Z Secret source: Actions 2024-08-21T01:24:29.0562998Z Prepare workflow directory 2024-08-21T01:24:29.3772542Z Prepare all required actions 2024-08-21T01:24:29.4036394Z Getting action download info 2024-08-21T01:24:29.6307428Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-21T01:24:30.4964927Z Download action repository 'pytorch/pytorch@main' (SHA:e8fc1e0118da85c3529f71fa60823fdbbbebee56) 2024-08-21T01:24:36.7516948Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-21T01:24:37.7612779Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-21T01:24:38.3700390Z Getting action download info 2024-08-21T01:24:38.4999566Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-21T01:24:39.0131733Z Getting action download info 2024-08-21T01:24:39.1367159Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-21T01:24:39.5944904Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-21T01:24:39.5948302Z ##[group] Inputs 2024-08-21T01:24:39.5948913Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-08-21T01:24:39.5950124Z test-matrix: {"include": [{"config": "inductor_torchbench_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "linux.gcp.a100"}]} 2024-08-21T01:24:39.5952314Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:24:39.5953918Z sync-tag: 2024-08-21T01:24:39.5954963Z timeout-minutes: 240 2024-08-21T01:24:39.5955433Z use-gha: anything-non-empty-to-use-gha 2024-08-21T01:24:39.5956038Z dashboard-tag: 2024-08-21T01:24:39.5956435Z s3-bucket: gha-artifacts 2024-08-21T01:24:39.5956868Z aws-role-to-assume: 2024-08-21T01:24:39.5957259Z ##[endgroup] 2024-08-21T01:24:39.5958495Z Complete job name: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-08-21T01:24:39.6701985Z A job started hook has been configured by the self-hosted runner administrator 2024-08-21T01:24:39.6894245Z ##[group]Run '/home/weiwangmeta/pre-job.sh' 2024-08-21T01:24:39.6912494Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:24:39.6913136Z ##[endgroup] 2024-08-21T01:24:39.7577464Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-21T01:24:39.7578188Z with: 2024-08-21T01:24:39.7578517Z submodules: recursive 2024-08-21T01:24:39.7578911Z fetch-depth: 0 2024-08-21T01:24:39.7579239Z env: 2024-08-21T01:24:39.7579547Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:24:39.7580032Z ##[endgroup] 2024-08-21T01:24:39.7804449Z ##[group]Run retry () { 2024-08-21T01:24:39.7805136Z retry () { 2024-08-21T01:24:39.7805824Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-21T01:24:39.7806506Z } 2024-08-21T01:24:39.7806846Z echo "${GITHUB_WORKSPACE}" 2024-08-21T01:24:39.7807348Z if [ -z "${NO_SUDO}" ]; then 2024-08-21T01:24:39.7807922Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-21T01:24:39.7808472Z else 2024-08-21T01:24:39.7808859Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-21T01:24:39.7809369Z fi 2024-08-21T01:24:39.7809776Z mkdir "${GITHUB_WORKSPACE}" 2024-08-21T01:24:39.7822744Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:24:39.7823382Z env: 2024-08-21T01:24:39.7823700Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:24:39.7824108Z NO_SUDO: 2024-08-21T01:24:39.7824410Z ##[endgroup] 2024-08-21T01:24:39.7895480Z /home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-08-21T01:24:42.6404503Z ##[group]Run malfet/checkout@silent-checkout 2024-08-21T01:24:42.6405092Z with: 2024-08-21T01:24:42.6405487Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:24:42.6406120Z fetch-depth: 0 2024-08-21T01:24:42.6406465Z submodules: recursive 2024-08-21T01:24:42.6406893Z quiet-checkout: true 2024-08-21T01:24:42.6407295Z repository: pytorch/pytorch 2024-08-21T01:24:42.6407923Z token: *** 2024-08-21T01:24:42.6408236Z ssh-strict: true 2024-08-21T01:24:42.6408615Z persist-credentials: true 2024-08-21T01:24:42.6409030Z clean: true 2024-08-21T01:24:42.6409398Z sparse-checkout-cone-mode: true 2024-08-21T01:24:42.6409859Z lfs: false 2024-08-21T01:24:42.6410199Z set-safe-directory: true 2024-08-21T01:24:42.6410594Z env: 2024-08-21T01:24:42.6410882Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:24:42.6411276Z ##[endgroup] 2024-08-21T01:24:42.7676011Z Syncing repository: pytorch/pytorch 2024-08-21T01:24:42.7678114Z ##[group]Getting Git version info 2024-08-21T01:24:42.7679132Z Working directory is '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch' 2024-08-21T01:24:42.7680245Z [command]/usr/bin/git version 2024-08-21T01:24:42.7680683Z git version 2.25.1 2024-08-21T01:24:42.7682459Z ##[endgroup] 2024-08-21T01:24:42.7697542Z Temporarily overriding HOME='/home/weiwangmeta/actions-runner/_work/_temp/ef7721bd-7b30-4457-afe5-f6cb20235170' before making global git config changes 2024-08-21T01:24:42.7699203Z Adding repository directory to the temporary git global config as a safe directory 2024-08-21T01:24:42.7703206Z [command]/usr/bin/git config --global --add safe.directory /home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-08-21T01:24:42.7736214Z Deleting the contents of '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch' 2024-08-21T01:24:42.7740265Z ##[group]Initializing the repository 2024-08-21T01:24:42.7743622Z [command]/usr/bin/git init /home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-08-21T01:24:42.7780270Z Initialized empty Git repository in /home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-21T01:24:42.7790532Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-21T01:24:42.7821149Z ##[endgroup] 2024-08-21T01:24:42.7821897Z ##[group]Disabling automatic garbage collection 2024-08-21T01:24:42.7824965Z [command]/usr/bin/git config --local gc.auto 0 2024-08-21T01:24:42.7851069Z ##[endgroup] 2024-08-21T01:24:42.7851687Z ##[group]Setting up auth 2024-08-21T01:24:42.7857340Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-21T01:24:42.7884021Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-21T01:24:42.8127617Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-21T01:24:42.8153571Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-21T01:24:42.8393011Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-21T01:24:42.8428403Z ##[endgroup] 2024-08-21T01:24:42.8429167Z ##[group]Fetching the repository 2024-08-21T01:24:42.8437441Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-21T01:24:45.7505385Z remote: Enumerating objects: 1008325 2024-08-21T01:24:45.7506713Z remote: Enumerating objects: 1010605, done. 2024-08-21T01:24:45.7508003Z remote: Counting objects: 0% (1/2280) 2024-08-21T01:24:45.7509099Z remote: Counting objects: 1% (23/2280) 2024-08-21T01:24:45.7509776Z remote: Counting objects: 2% (46/2280) 2024-08-21T01:24:45.7510776Z remote: Counting objects: 3% (69/2280) 2024-08-21T01:24:45.7511682Z remote: Counting objects: 4% (92/2280) 2024-08-21T01:24:45.7512801Z remote: Counting objects: 5% (114/2280) 2024-08-21T01:24:45.7513762Z remote: Counting objects: 6% (137/2280) 2024-08-21T01:24:45.7514669Z remote: Counting objects: 7% (160/2280) 2024-08-21T01:24:45.7515611Z remote: Counting objects: 8% (183/2280) 2024-08-21T01:24:45.7516735Z remote: Counting objects: 9% (206/2280) 2024-08-21T01:24:45.7517952Z remote: Counting objects: 10% (228/2280) 2024-08-21T01:24:45.7518975Z remote: Counting objects: 11% (251/2280) 2024-08-21T01:24:45.7520068Z remote: Counting objects: 12% (274/2280) 2024-08-21T01:24:45.7521192Z remote: Counting objects: 13% (297/2280) 2024-08-21T01:24:45.7522353Z remote: Counting objects: 14% (320/2280) 2024-08-21T01:24:45.7523323Z remote: Counting objects: 15% (342/2280) 2024-08-21T01:24:45.7524388Z remote: Counting objects: 16% (365/2280) 2024-08-21T01:24:45.7525059Z remote: Counting objects: 17% (388/2280) 2024-08-21T01:24:45.7525734Z remote: Counting objects: 18% (411/2280) 2024-08-21T01:24:45.7526341Z remote: Counting objects: 19% (434/2280) 2024-08-21T01:24:45.7527121Z remote: Counting objects: 20% (456/2280) 2024-08-21T01:24:45.7528084Z remote: Counting objects: 21% (479/2280) 2024-08-21T01:24:45.7530219Z remote: Counting objects: 22% (502/2280) 2024-08-21T01:24:45.7531251Z remote: Counting objects: 23% (525/2280) 2024-08-21T01:24:45.7531980Z remote: Counting objects: 24% (548/2280) 2024-08-21T01:24:45.7533081Z remote: Counting objects: 25% (570/2280) 2024-08-21T01:24:45.7533833Z remote: Counting objects: 26% (593/2280) 2024-08-21T01:24:45.7534519Z remote: Counting objects: 27% (616/2280) 2024-08-21T01:24:45.7535497Z remote: Counting objects: 28% (639/2280) 2024-08-21T01:24:45.7536164Z remote: Counting objects: 29% (662/2280) 2024-08-21T01:24:45.7536794Z remote: Counting objects: 30% (684/2280) 2024-08-21T01:24:45.7537611Z remote: Counting objects: 31% (707/2280) 2024-08-21T01:24:45.7538376Z remote: Counting objects: 32% (730/2280) 2024-08-21T01:24:45.7538976Z remote: Counting objects: 33% (753/2280) 2024-08-21T01:24:45.7539596Z remote: Counting objects: 34% (776/2280) 2024-08-21T01:24:45.7540245Z remote: Counting objects: 35% (798/2280) 2024-08-21T01:24:45.7540864Z remote: Counting objects: 36% (821/2280) 2024-08-21T01:24:45.7541968Z remote: Counting objects: 37% (844/2280) 2024-08-21T01:24:45.7542617Z remote: Counting objects: 38% (867/2280) 2024-08-21T01:24:45.7543233Z remote: Counting objects: 39% (890/2280) 2024-08-21T01:24:45.7543838Z remote: Counting objects: 40% (912/2280) 2024-08-21T01:24:45.7544429Z remote: Counting objects: 41% (935/2280) 2024-08-21T01:24:45.7545042Z remote: Counting objects: 42% (958/2280) 2024-08-21T01:24:45.7545917Z remote: Counting objects: 43% (981/2280) 2024-08-21T01:24:45.7546543Z remote: Counting objects: 44% (1004/2280) 2024-08-21T01:24:45.7547169Z remote: Counting objects: 45% (1026/2280) 2024-08-21T01:24:45.7548173Z remote: Counting objects: 46% (1049/2280) 2024-08-21T01:24:45.7548819Z remote: Counting objects: 47% (1072/2280) 2024-08-21T01:24:45.7549453Z remote: Counting objects: 48% (1095/2280) 2024-08-21T01:24:45.7550077Z remote: Counting objects: 49% (1118/2280) 2024-08-21T01:24:45.7550717Z remote: Counting objects: 50% (1140/2280) 2024-08-21T01:24:45.7551347Z remote: Counting objects: 51% (1163/2280) 2024-08-21T01:24:45.7551964Z remote: Counting objects: 52% (1186/2280) 2024-08-21T01:24:45.7552598Z remote: Counting objects: 53% (1209/2280) 2024-08-21T01:24:45.7553224Z remote: Counting objects: 54% (1232/2280) 2024-08-21T01:24:45.7553857Z remote: Counting objects: 55% (1254/2280) 2024-08-21T01:24:45.7554480Z remote: Counting objects: 56% (1277/2280) 2024-08-21T01:24:45.7555108Z remote: Counting objects: 57% (1300/2280) 2024-08-21T01:24:45.7555887Z remote: Counting objects: 58% (1323/2280) 2024-08-21T01:24:45.7556571Z remote: Counting objects: 59% (1346/2280) 2024-08-21T01:24:45.7557201Z remote: Counting objects: 60% (1368/2280) 2024-08-21T01:24:45.7557818Z remote: Counting objects: 61% (1391/2280) 2024-08-21T01:24:45.7558523Z remote: Counting objects: 62% (1414/2280) 2024-08-21T01:24:45.7559161Z remote: Counting objects: 63% (1437/2280) 2024-08-21T01:24:45.7559795Z remote: Counting objects: 64% (1460/2280) 2024-08-21T01:24:45.7560407Z remote: Counting objects: 65% (1482/2280) 2024-08-21T01:24:45.7561033Z remote: Counting objects: 66% (1505/2280) 2024-08-21T01:24:45.7561662Z remote: Counting objects: 67% (1528/2280) 2024-08-21T01:24:45.7562293Z remote: Counting objects: 68% (1551/2280) 2024-08-21T01:24:45.7562909Z remote: Counting objects: 69% (1574/2280) 2024-08-21T01:24:45.7563539Z remote: Counting objects: 70% (1596/2280) 2024-08-21T01:24:45.7564168Z remote: Counting objects: 71% (1619/2280) 2024-08-21T01:24:45.7564797Z remote: Counting objects: 72% (1642/2280) 2024-08-21T01:24:45.7565408Z remote: Counting objects: 73% (1665/2280) 2024-08-21T01:24:45.7567096Z remote: Counting objects: 74% (1688/2280) 2024-08-21T01:24:45.7567773Z remote: Counting objects: 75% (1710/2280) 2024-08-21T01:24:45.7568461Z remote: Counting objects: 76% (1733/2280) 2024-08-21T01:24:45.7569086Z remote: Counting objects: 77% (1756/2280) 2024-08-21T01:24:45.7569717Z remote: Counting objects: 78% (1779/2280) 2024-08-21T01:24:45.7570354Z remote: Counting objects: 79% (1802/2280) 2024-08-21T01:24:45.7570980Z remote: Counting objects: 80% (1824/2280) 2024-08-21T01:24:45.7571607Z remote: Counting objects: 81% (1847/2280) 2024-08-21T01:24:45.7572239Z remote: Counting objects: 82% (1870/2280) 2024-08-21T01:24:45.7572867Z remote: Counting objects: 83% (1893/2280) 2024-08-21T01:24:45.7573494Z remote: Counting objects: 84% (1916/2280) 2024-08-21T01:24:45.7574108Z remote: Counting objects: 85% (1938/2280) 2024-08-21T01:24:45.7574734Z remote: Counting objects: 86% (1961/2280) 2024-08-21T01:24:45.7575359Z remote: Counting objects: 87% (1984/2280) 2024-08-21T01:24:45.7576403Z remote: Counting objects: 88% (2007/2280) 2024-08-21T01:24:45.7577024Z remote: Counting objects: 89% (2030/2280) 2024-08-21T01:24:45.7577653Z remote: Counting objects: 90% (2052/2280) 2024-08-21T01:24:45.7578306Z remote: Counting objects: 91% (2075/2280) 2024-08-21T01:24:45.7578941Z remote: Counting objects: 92% (2098/2280) 2024-08-21T01:24:45.7579556Z remote: Counting objects: 93% (2121/2280) 2024-08-21T01:24:45.7580313Z remote: Counting objects: 94% (2144/2280) 2024-08-21T01:24:45.7580964Z remote: Counting objects: 95% (2166/2280) 2024-08-21T01:24:45.7581592Z remote: Counting objects: 96% (2189/2280) 2024-08-21T01:24:45.7582219Z remote: Counting objects: 97% (2212/2280) 2024-08-21T01:24:45.7582830Z remote: Counting objects: 98% (2235/2280) 2024-08-21T01:24:45.7583462Z remote: Counting objects: 99% (2258/2280) 2024-08-21T01:24:45.7584088Z remote: Counting objects: 100% (2280/2280) 2024-08-21T01:24:45.7584767Z remote: Counting objects: 100% (2280/2280), done. 2024-08-21T01:24:45.7669721Z remote: Compressing objects: 0% (1/1087) 2024-08-21T01:24:45.7886978Z remote: Compressing objects: 1% (11/1087) 2024-08-21T01:24:45.8124714Z remote: Compressing objects: 2% (22/1087) 2024-08-21T01:24:45.8696619Z remote: Compressing objects: 3% (33/1087) 2024-08-21T01:24:45.9416138Z remote: Compressing objects: 4% (44/1087) 2024-08-21T01:24:45.9943081Z remote: Compressing objects: 5% (55/1087) 2024-08-21T01:24:46.0374744Z remote: Compressing objects: 6% (66/1087) 2024-08-21T01:24:46.0757318Z remote: Compressing objects: 7% (77/1087) 2024-08-21T01:24:46.1029530Z remote: Compressing objects: 8% (87/1087) 2024-08-21T01:24:46.1322316Z remote: Compressing objects: 9% (98/1087) 2024-08-21T01:24:46.1491160Z remote: Compressing objects: 10% (109/1087) 2024-08-21T01:24:46.1676573Z remote: Compressing objects: 11% (120/1087) 2024-08-21T01:24:46.1907889Z remote: Compressing objects: 12% (131/1087) 2024-08-21T01:24:46.2050667Z remote: Compressing objects: 13% (142/1087) 2024-08-21T01:24:46.2151283Z remote: Compressing objects: 14% (153/1087) 2024-08-21T01:24:46.2222180Z remote: Compressing objects: 15% (164/1087) 2024-08-21T01:24:46.2295443Z remote: Compressing objects: 16% (174/1087) 2024-08-21T01:24:46.2351645Z remote: Compressing objects: 17% (185/1087) 2024-08-21T01:24:46.2413902Z remote: Compressing objects: 18% (196/1087) 2024-08-21T01:24:46.2442832Z remote: Compressing objects: 19% (207/1087) 2024-08-21T01:24:46.2471436Z remote: Compressing objects: 20% (218/1087) 2024-08-21T01:24:46.2480762Z remote: Compressing objects: 21% (229/1087) 2024-08-21T01:24:46.2487650Z remote: Compressing objects: 22% (240/1087) 2024-08-21T01:24:46.2495365Z remote: Compressing objects: 23% (251/1087) 2024-08-21T01:24:46.2502023Z remote: Compressing objects: 24% (261/1087) 2024-08-21T01:24:46.2511327Z remote: Compressing objects: 25% (272/1087) 2024-08-21T01:24:46.2533540Z remote: Compressing objects: 26% (283/1087) 2024-08-21T01:24:46.2554531Z remote: Compressing objects: 27% (294/1087) 2024-08-21T01:24:46.2563507Z remote: Compressing objects: 28% (305/1087) 2024-08-21T01:24:46.2577445Z remote: Compressing objects: 29% (316/1087) 2024-08-21T01:24:46.2584704Z remote: Compressing objects: 30% (327/1087) 2024-08-21T01:24:46.2600351Z remote: Compressing objects: 31% (337/1087) 2024-08-21T01:24:46.2607782Z remote: Compressing objects: 32% (348/1087) 2024-08-21T01:24:46.2614987Z remote: Compressing objects: 33% (359/1087) 2024-08-21T01:24:46.2624396Z remote: Compressing objects: 34% (370/1087) 2024-08-21T01:24:46.2637768Z remote: Compressing objects: 35% (381/1087) 2024-08-21T01:24:46.2645511Z remote: Compressing objects: 36% (392/1087) 2024-08-21T01:24:46.2654072Z remote: Compressing objects: 37% (403/1087) 2024-08-21T01:24:46.2660357Z remote: Compressing objects: 38% (414/1087) 2024-08-21T01:24:46.2670058Z remote: Compressing objects: 39% (424/1087) 2024-08-21T01:24:46.2678760Z remote: Compressing objects: 40% (435/1087) 2024-08-21T01:24:46.2684542Z remote: Compressing objects: 41% (446/1087) 2024-08-21T01:24:46.2690010Z remote: Compressing objects: 42% (457/1087) 2024-08-21T01:24:46.2696630Z remote: Compressing objects: 43% (468/1087) 2024-08-21T01:24:46.2703455Z remote: Compressing objects: 44% (479/1087) 2024-08-21T01:24:46.2709595Z remote: Compressing objects: 45% (490/1087) 2024-08-21T01:24:46.2711593Z remote: Compressing objects: 46% (501/1087) 2024-08-21T01:24:46.2713253Z remote: Compressing objects: 47% (511/1087) 2024-08-21T01:24:46.2717797Z remote: Compressing objects: 48% (522/1087) 2024-08-21T01:24:46.2719548Z remote: Compressing objects: 49% (533/1087) 2024-08-21T01:24:46.2724152Z remote: Compressing objects: 50% (544/1087) 2024-08-21T01:24:46.2725301Z remote: Compressing objects: 51% (555/1087) 2024-08-21T01:24:46.2726882Z remote: Compressing objects: 52% (566/1087) 2024-08-21T01:24:46.2728014Z remote: Compressing objects: 53% (577/1087) 2024-08-21T01:24:46.2730077Z remote: Compressing objects: 54% (587/1087) 2024-08-21T01:24:46.2731310Z remote: Compressing objects: 55% (598/1087) 2024-08-21T01:24:46.2732624Z remote: Compressing objects: 56% (609/1087) 2024-08-21T01:24:46.2733834Z remote: Compressing objects: 57% (620/1087) 2024-08-21T01:24:46.2735035Z remote: Compressing objects: 58% (631/1087) 2024-08-21T01:24:46.2736032Z remote: Compressing objects: 59% (642/1087) 2024-08-21T01:24:46.2736676Z remote: Compressing objects: 60% (653/1087) 2024-08-21T01:24:46.2737332Z remote: Compressing objects: 61% (664/1087) 2024-08-21T01:24:46.2738000Z remote: Compressing objects: 62% (674/1087) 2024-08-21T01:24:46.2738764Z remote: Compressing objects: 63% (685/1087) 2024-08-21T01:24:46.2741307Z remote: Compressing objects: 64% (696/1087) 2024-08-21T01:24:46.2746001Z remote: Compressing objects: 65% (707/1087) 2024-08-21T01:24:46.2750311Z remote: Compressing objects: 66% (718/1087) 2024-08-21T01:24:46.2755370Z remote: Compressing objects: 67% (729/1087) 2024-08-21T01:24:46.2761485Z remote: Compressing objects: 68% (740/1087) 2024-08-21T01:24:46.2764844Z remote: Compressing objects: 69% (751/1087) 2024-08-21T01:24:46.2769027Z remote: Compressing objects: 70% (761/1087) 2024-08-21T01:24:46.2770070Z remote: Compressing objects: 71% (772/1087) 2024-08-21T01:24:46.2773836Z remote: Compressing objects: 72% (783/1087) 2024-08-21T01:24:46.2778015Z remote: Compressing objects: 73% (794/1087) 2024-08-21T01:24:46.2781135Z remote: Compressing objects: 74% (805/1087) 2024-08-21T01:24:46.2787129Z remote: Compressing objects: 75% (816/1087) 2024-08-21T01:24:46.2788168Z remote: Compressing objects: 76% (827/1087) 2024-08-21T01:24:46.2789298Z remote: Compressing objects: 77% (837/1087) 2024-08-21T01:24:46.2791623Z remote: Compressing objects: 78% (848/1087) 2024-08-21T01:24:46.2794676Z remote: Compressing objects: 79% (859/1087) 2024-08-21T01:24:46.2796880Z remote: Compressing objects: 80% (870/1087) 2024-08-21T01:24:46.2798786Z remote: Compressing objects: 81% (881/1087) 2024-08-21T01:24:46.2801413Z remote: Compressing objects: 82% (892/1087) 2024-08-21T01:24:46.2802475Z remote: Compressing objects: 83% (903/1087) 2024-08-21T01:24:46.2804376Z remote: Compressing objects: 84% (914/1087) 2024-08-21T01:24:46.2805944Z remote: Compressing objects: 85% (924/1087) 2024-08-21T01:24:46.2807722Z remote: Compressing objects: 86% (935/1087) 2024-08-21T01:24:46.2810836Z remote: Compressing objects: 87% (946/1087) 2024-08-21T01:24:46.2812752Z remote: Compressing objects: 88% (957/1087) 2024-08-21T01:24:46.2814641Z remote: Compressing objects: 89% (968/1087) 2024-08-21T01:24:46.2816608Z remote: Compressing objects: 90% (979/1087) 2024-08-21T01:24:46.2817735Z remote: Compressing objects: 91% (990/1087) 2024-08-21T01:24:46.2818763Z remote: Compressing objects: 92% (1001/1087) 2024-08-21T01:24:46.2819835Z remote: Compressing objects: 93% (1011/1087) 2024-08-21T01:24:46.2821296Z remote: Compressing objects: 94% (1022/1087) 2024-08-21T01:24:46.2823309Z remote: Compressing objects: 95% (1033/1087) 2024-08-21T01:24:46.2824578Z remote: Compressing objects: 96% (1044/1087) 2024-08-21T01:24:46.2826318Z remote: Compressing objects: 97% (1055/1087) 2024-08-21T01:24:46.2827909Z remote: Compressing objects: 98% (1066/1087) 2024-08-21T01:24:46.2829157Z remote: Compressing objects: 99% (1077/1087) 2024-08-21T01:24:46.2829932Z remote: Compressing objects: 100% (1087/1087) 2024-08-21T01:24:46.2830657Z remote: Compressing objects: 100% (1087/1087), done. 2024-08-21T01:25:10.6918983Z remote: Total 1010605 (delta 1508), reused 1828 (delta 1187), pack-reused 1008325 (from 1) 2024-08-21T01:25:45.4808302Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-21T01:25:45.4834029Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:25:45.4839658Z ##[endgroup] 2024-08-21T01:25:45.4840406Z ##[group]Determining the checkout info 2024-08-21T01:25:45.4841287Z ##[endgroup] 2024-08-21T01:25:45.4841893Z ##[group]Checking out the ref 2024-08-21T01:25:45.4846473Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:25:47.0910511Z ##[endgroup] 2024-08-21T01:25:47.0913690Z ##[group]Setting up auth for fetching submodules 2024-08-21T01:25:47.0918219Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-21T01:25:47.0961702Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-21T01:25:47.0989596Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-21T01:25:47.1017963Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-21T01:25:47.1051064Z ##[endgroup] 2024-08-21T01:25:47.1051758Z ##[group]Fetching submodules 2024-08-21T01:25:47.1054869Z [command]/usr/bin/git submodule sync --recursive 2024-08-21T01:25:47.1345299Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-21T01:25:47.1621641Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-21T01:25:47.1628121Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-21T01:25:47.1634350Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-21T01:25:47.1639982Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-21T01:25:47.1646840Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-21T01:25:47.1652911Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-21T01:25:47.1659119Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-21T01:25:47.1665085Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-21T01:25:47.1670988Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-21T01:25:47.1676294Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-21T01:25:47.1681609Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-21T01:25:47.1687214Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-21T01:25:47.1692874Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-21T01:25:47.1698666Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-21T01:25:47.1704000Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-21T01:25:47.1710198Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-21T01:25:47.1715476Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-21T01:25:47.1721317Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-21T01:25:47.1726824Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-21T01:25:47.1732997Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-21T01:25:47.1739013Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-21T01:25:47.1745169Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-21T01:25:47.1751688Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-21T01:25:47.1759303Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-21T01:25:47.1765157Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-21T01:25:47.1771901Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-21T01:25:47.1778376Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-21T01:25:47.1784976Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-21T01:25:47.1791909Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-21T01:25:47.1798992Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-21T01:25:47.1805283Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-21T01:25:47.1812445Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-21T01:25:47.1818646Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-21T01:25:47.1825544Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-21T01:25:47.1913317Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-21T01:25:47.7755320Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-21T01:25:48.1690635Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-21T01:25:48.5164267Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-21T01:25:49.0776111Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-21T01:25:52.3945815Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-21T01:26:09.9840168Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-21T01:26:10.6862279Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-21T01:26:11.6962788Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-21T01:26:12.6041267Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-21T01:26:14.0474642Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-21T01:26:16.6537091Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-21T01:26:22.1873553Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-21T01:26:24.1705267Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-21T01:26:26.3648077Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-21T01:26:28.2254236Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-21T01:26:28.9390828Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-21T01:26:29.5605657Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-21T01:26:31.2826795Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-21T01:26:31.9422071Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-21T01:26:32.5232069Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-21T01:26:34.4684404Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-21T01:26:35.5426690Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-21T01:26:36.5101346Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-21T01:26:45.4505433Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-21T01:26:48.8668749Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-21T01:27:01.2680409Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-21T01:27:01.7660433Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-21T01:27:13.3740606Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-21T01:27:13.7270266Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-21T01:27:14.1670889Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-21T01:27:15.4230915Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-21T01:27:16.0091697Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-21T01:27:16.9764240Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-21T01:27:17.6891622Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-21T01:27:17.7282586Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-21T01:27:17.7643174Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-21T01:27:17.8172649Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-21T01:27:17.8902804Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-21T01:27:19.0088963Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-21T01:27:19.0622965Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-21T01:27:19.1426426Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-21T01:27:19.2847297Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-21T01:27:19.3505971Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-21T01:27:19.9478699Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-21T01:27:20.2667204Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-21T01:27:20.3868513Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-21T01:27:20.3920863Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-21T01:27:20.3925002Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-21T01:27:20.3931401Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-21T01:27:20.3936626Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-21T01:27:20.3942450Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-21T01:27:20.3990332Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-21T01:27:21.7017581Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-21T01:27:22.6189538Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-21T01:27:25.0215718Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-21T01:27:26.7051741Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-21T01:27:27.3457420Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-21T01:27:27.4868956Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-21T01:27:27.9582316Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-21T01:27:28.0530857Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-21T01:27:28.0894552Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-21T01:27:28.2404040Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-21T01:27:28.3153996Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-21T01:27:28.3871843Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-21T01:27:28.4404093Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-21T01:27:28.5172572Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-21T01:27:28.5578505Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-21T01:27:28.5626159Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-21T01:27:28.5670319Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-21T01:27:45.0018131Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-21T01:27:45.0475665Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-21T01:27:45.1805175Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-21T01:27:45.1862162Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-21T01:27:45.1868845Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-21T01:27:45.1875064Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-21T01:27:45.1925023Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-21T01:27:46.1728146Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-21T01:27:47.9946054Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-21T01:27:49.7880160Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-21T01:27:49.7934093Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-21T01:27:49.7939229Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-21T01:27:49.7944206Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-21T01:27:49.7950377Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-21T01:27:49.7955087Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-21T01:27:49.7961081Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-21T01:27:49.7965990Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-21T01:27:49.7970735Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-21T01:27:49.8019591Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-21T01:27:51.1424005Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-21T01:27:51.8032398Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-21T01:27:53.6427995Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-21T01:27:54.2486766Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-21T01:27:55.1590583Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-21T01:27:56.8459588Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-21T01:28:04.6795665Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-21T01:28:05.3675829Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-21T01:28:05.4127005Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-21T01:28:05.4774986Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-21T01:28:05.5144958Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-21T01:28:05.5197250Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-21T01:28:05.5242796Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-21T01:28:06.1853669Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-21T01:28:06.2300255Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-21T01:28:06.2983435Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-21T01:28:06.4377603Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-21T01:28:06.4784551Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-21T01:28:06.5468500Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-21T01:28:06.6365652Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-21T01:28:06.7062279Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-21T01:28:06.7589351Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-21T01:28:06.9029854Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-21T01:28:07.3058943Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-21T01:28:07.3137855Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-21T01:28:07.3143416Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-21T01:28:07.3215676Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-21T01:28:08.1055801Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-21T01:28:09.4059881Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-21T01:28:09.4691744Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-21T01:28:09.5656784Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-21T01:28:09.5710012Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-21T01:28:09.5716720Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-21T01:28:09.5722509Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-21T01:28:09.5727670Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-21T01:28:09.5734755Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-21T01:28:09.5740243Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-21T01:28:09.5746379Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-21T01:28:09.5751796Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-21T01:28:09.5802201Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-21T01:28:10.3329003Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-21T01:28:12.0052587Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-21T01:28:12.6078213Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-21T01:28:20.8858670Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-21T01:28:21.4389355Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-21T01:28:21.8951684Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-21T01:28:22.4816922Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-21T01:28:32.5483887Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-21T01:28:32.6168190Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-21T01:28:32.6568748Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-21T01:28:32.7905519Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-21T01:28:32.8278865Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-21T01:28:32.8653262Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-21T01:28:32.9038579Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-21T01:28:32.9087608Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-21T01:28:32.9092643Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-21T01:28:32.9139089Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-21T01:28:36.2030878Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-21T01:28:38.2303400Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-21T01:28:38.3050591Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-21T01:28:38.8117848Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-21T01:28:38.8539516Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-21T01:28:39.1826713Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-21T01:28:39.1885744Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-21T01:28:39.1890448Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-21T01:28:39.1942920Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-21T01:28:39.9446847Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-21T01:28:41.6653235Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-21T01:28:41.7727619Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-21T01:28:41.8092015Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-21T01:28:41.8497713Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-21T01:28:41.9187059Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-21T01:28:41.9800873Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-21T01:28:42.0559875Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-21T01:28:42.1102808Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-21T01:28:42.1152086Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-21T01:28:42.1156987Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-21T01:28:42.1161808Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-21T01:28:42.1167367Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-21T01:28:42.1215231Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-21T01:28:43.8487490Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-21T01:28:44.3357105Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-21T01:28:45.9545418Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-21T01:28:47.3044470Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-21T01:28:47.3444335Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-21T01:28:47.4487675Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-21T01:28:47.5026624Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-21T01:28:47.5081313Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-21T01:28:47.5129272Z Cloning into '/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-21T01:28:47.9701332Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-21T01:28:47.9804069Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-21T01:28:48.0085104Z Entering 'android/libs/fbjni' 2024-08-21T01:28:48.0126329Z Entering 'third_party/FP16' 2024-08-21T01:28:48.0169534Z Entering 'third_party/FXdiv' 2024-08-21T01:28:48.0209969Z Entering 'third_party/NNPACK' 2024-08-21T01:28:48.0250756Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-21T01:28:48.0291303Z Entering 'third_party/XNNPACK' 2024-08-21T01:28:48.0352365Z Entering 'third_party/benchmark' 2024-08-21T01:28:48.0393943Z Entering 'third_party/cpp-httplib' 2024-08-21T01:28:48.0434872Z Entering 'third_party/cpuinfo' 2024-08-21T01:28:48.0475547Z Entering 'third_party/cudnn_frontend' 2024-08-21T01:28:48.0516897Z Entering 'third_party/cutlass' 2024-08-21T01:28:48.0567066Z Entering 'third_party/eigen' 2024-08-21T01:28:48.0609920Z Entering 'third_party/fbgemm' 2024-08-21T01:28:48.0652801Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-21T01:28:48.0692605Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-21T01:28:48.0731892Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-21T01:28:48.0779307Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-21T01:28:48.0818531Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-21T01:28:48.0860726Z Entering 'third_party/flatbuffers' 2024-08-21T01:28:48.0904785Z Entering 'third_party/fmt' 2024-08-21T01:28:48.0944996Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-21T01:28:48.0984873Z Entering 'third_party/gloo' 2024-08-21T01:28:48.1024784Z Entering 'third_party/googletest' 2024-08-21T01:28:48.1064691Z Entering 'third_party/ideep' 2024-08-21T01:28:48.1103734Z Entering 'third_party/ideep/mkl-dnn' 2024-08-21T01:28:48.1152055Z Entering 'third_party/ittapi' 2024-08-21T01:28:48.1192218Z Entering 'third_party/kineto' 2024-08-21T01:28:48.1233194Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-21T01:28:48.1273663Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-21T01:28:48.1313877Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-21T01:28:48.1354688Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-21T01:28:48.1394228Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-21T01:28:48.1432747Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-21T01:28:48.1474864Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-21T01:28:48.1514168Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-21T01:28:48.1553682Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-21T01:28:48.1593183Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-21T01:28:48.1633853Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-21T01:28:48.1672950Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-21T01:28:48.1713821Z Entering 'third_party/mimalloc' 2024-08-21T01:28:48.1754874Z Entering 'third_party/nccl/nccl' 2024-08-21T01:28:48.1795598Z Entering 'third_party/nlohmann' 2024-08-21T01:28:48.1838003Z Entering 'third_party/onnx' 2024-08-21T01:28:48.1897756Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-21T01:28:48.1937836Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-21T01:28:48.1980213Z Entering 'third_party/opentelemetry-cpp' 2024-08-21T01:28:48.2023054Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-21T01:28:48.2062543Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-21T01:28:48.2101205Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-21T01:28:48.2140192Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-21T01:28:48.2180695Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-21T01:28:48.2219861Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-21T01:28:48.2258401Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-21T01:28:48.2295593Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-21T01:28:48.2338354Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-21T01:28:48.2380774Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-21T01:28:48.2443769Z Entering 'third_party/pocketfft' 2024-08-21T01:28:48.2484974Z Entering 'third_party/protobuf' 2024-08-21T01:28:48.2528610Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-21T01:28:48.2566671Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-21T01:28:48.2608534Z Entering 'third_party/psimd' 2024-08-21T01:28:48.2648516Z Entering 'third_party/pthreadpool' 2024-08-21T01:28:48.2688716Z Entering 'third_party/pybind11' 2024-08-21T01:28:48.2728890Z Entering 'third_party/python-peachpy' 2024-08-21T01:28:48.2769921Z Entering 'third_party/sleef' 2024-08-21T01:28:48.2810208Z Entering 'third_party/tensorpipe' 2024-08-21T01:28:48.2851216Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-21T01:28:48.2890430Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-21T01:28:48.2930180Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-21T01:28:48.2969271Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-21T01:28:48.3008152Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-21T01:28:48.3064933Z ##[endgroup] 2024-08-21T01:28:48.3066226Z ##[group]Persisting credentials for submodules 2024-08-21T01:28:48.3088954Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-21T01:28:48.3361116Z Entering 'android/libs/fbjni' 2024-08-21T01:28:48.3408462Z Entering 'third_party/FP16' 2024-08-21T01:28:48.3454935Z Entering 'third_party/FXdiv' 2024-08-21T01:28:48.3501055Z Entering 'third_party/NNPACK' 2024-08-21T01:28:48.3547579Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-21T01:28:48.3593711Z Entering 'third_party/XNNPACK' 2024-08-21T01:28:48.3659883Z Entering 'third_party/benchmark' 2024-08-21T01:28:48.3706056Z Entering 'third_party/cpp-httplib' 2024-08-21T01:28:48.3754032Z Entering 'third_party/cpuinfo' 2024-08-21T01:28:48.3801142Z Entering 'third_party/cudnn_frontend' 2024-08-21T01:28:48.3847415Z Entering 'third_party/cutlass' 2024-08-21T01:28:48.3902366Z Entering 'third_party/eigen' 2024-08-21T01:28:48.3951735Z Entering 'third_party/fbgemm' 2024-08-21T01:28:48.3997215Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-21T01:28:48.4040966Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-21T01:28:48.4085964Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-21T01:28:48.4139265Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-21T01:28:48.4184163Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-21T01:28:48.4231138Z Entering 'third_party/flatbuffers' 2024-08-21T01:28:48.4279869Z Entering 'third_party/fmt' 2024-08-21T01:28:48.4324718Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-21T01:28:48.4371519Z Entering 'third_party/gloo' 2024-08-21T01:28:48.4448755Z Entering 'third_party/googletest' 2024-08-21T01:28:48.4464884Z Entering 'third_party/ideep' 2024-08-21T01:28:48.4510656Z Entering 'third_party/ideep/mkl-dnn' 2024-08-21T01:28:48.4566151Z Entering 'third_party/ittapi' 2024-08-21T01:28:48.4612820Z Entering 'third_party/kineto' 2024-08-21T01:28:48.4658566Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-21T01:28:48.4703270Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-21T01:28:48.4749677Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-21T01:28:48.4794111Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-21T01:28:48.4840213Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-21T01:28:48.4885301Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-21T01:28:48.4933777Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-21T01:28:48.4978370Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-21T01:28:48.5024155Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-21T01:28:48.5071538Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-21T01:28:48.5118361Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-21T01:28:48.5162109Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-21T01:28:48.5207716Z Entering 'third_party/mimalloc' 2024-08-21T01:28:48.5254239Z Entering 'third_party/nccl/nccl' 2024-08-21T01:28:48.5300028Z Entering 'third_party/nlohmann' 2024-08-21T01:28:48.5347033Z Entering 'third_party/onnx' 2024-08-21T01:28:48.5412810Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-21T01:28:48.5458555Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-21T01:28:48.5507364Z Entering 'third_party/opentelemetry-cpp' 2024-08-21T01:28:48.5555662Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-21T01:28:48.5600414Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-21T01:28:48.5645734Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-21T01:28:48.5691781Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-21T01:28:48.5738372Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-21T01:28:48.5783208Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-21T01:28:48.5830413Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-21T01:28:48.5874584Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-21T01:28:48.5922723Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-21T01:28:48.5972289Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-21T01:28:48.6043763Z Entering 'third_party/pocketfft' 2024-08-21T01:28:48.6091254Z Entering 'third_party/protobuf' 2024-08-21T01:28:48.6141790Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-21T01:28:48.6185955Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-21T01:28:48.6234315Z Entering 'third_party/psimd' 2024-08-21T01:28:48.6280636Z Entering 'third_party/pthreadpool' 2024-08-21T01:28:48.6326332Z Entering 'third_party/pybind11' 2024-08-21T01:28:48.6374321Z Entering 'third_party/python-peachpy' 2024-08-21T01:28:48.6420589Z Entering 'third_party/sleef' 2024-08-21T01:28:48.6467847Z Entering 'third_party/tensorpipe' 2024-08-21T01:28:48.6514979Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-21T01:28:48.6562697Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-21T01:28:48.6606507Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-21T01:28:48.6653472Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-21T01:28:48.6697227Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-21T01:28:48.6763148Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-21T01:28:48.7033003Z Entering 'android/libs/fbjni' 2024-08-21T01:28:48.7077364Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-21T01:28:48.7098018Z Entering 'third_party/FP16' 2024-08-21T01:28:48.7143691Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-21T01:28:48.7162638Z Entering 'third_party/FXdiv' 2024-08-21T01:28:48.7205554Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-21T01:28:48.7224309Z Entering 'third_party/NNPACK' 2024-08-21T01:28:48.7268238Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-21T01:28:48.7287761Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-21T01:28:48.7331237Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-21T01:28:48.7350270Z Entering 'third_party/XNNPACK' 2024-08-21T01:28:48.7392492Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-21T01:28:48.7431798Z Entering 'third_party/benchmark' 2024-08-21T01:28:48.7473979Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-21T01:28:48.7494178Z Entering 'third_party/cpp-httplib' 2024-08-21T01:28:48.7537044Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-21T01:28:48.7556795Z Entering 'third_party/cpuinfo' 2024-08-21T01:28:48.7600645Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-21T01:28:48.7620614Z Entering 'third_party/cudnn_frontend' 2024-08-21T01:28:48.7664167Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-21T01:28:48.7683559Z Entering 'third_party/cutlass' 2024-08-21T01:28:48.7726792Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-21T01:28:48.7757777Z Entering 'third_party/eigen' 2024-08-21T01:28:48.7801107Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-21T01:28:48.7822669Z Entering 'third_party/fbgemm' 2024-08-21T01:28:48.7867064Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-21T01:28:48.7886274Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-21T01:28:48.7928917Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-21T01:28:48.7947135Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-21T01:28:48.7990444Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-21T01:28:48.8009104Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-21T01:28:48.8051387Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-21T01:28:48.8079804Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-21T01:28:48.8122442Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-21T01:28:48.8141788Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-21T01:28:48.8184564Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-21T01:28:48.8204339Z Entering 'third_party/flatbuffers' 2024-08-21T01:28:48.8248706Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-21T01:28:48.8270902Z Entering 'third_party/fmt' 2024-08-21T01:28:48.8313544Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-21T01:28:48.8333082Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-21T01:28:48.8375790Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-21T01:28:48.8395052Z Entering 'third_party/gloo' 2024-08-21T01:28:48.8437159Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-21T01:28:48.8456250Z Entering 'third_party/googletest' 2024-08-21T01:28:48.8498534Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-21T01:28:48.8518031Z Entering 'third_party/ideep' 2024-08-21T01:28:48.8560399Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-21T01:28:48.8578209Z Entering 'third_party/ideep/mkl-dnn' 2024-08-21T01:28:48.8619730Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-21T01:28:48.8647017Z Entering 'third_party/ittapi' 2024-08-21T01:28:48.8689536Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-21T01:28:48.8708863Z Entering 'third_party/kineto' 2024-08-21T01:28:48.8750694Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-21T01:28:48.8769583Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-21T01:28:48.8811915Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-21T01:28:48.8830605Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-21T01:28:48.8874490Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-21T01:28:48.8894330Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-21T01:28:48.8938079Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-21T01:28:48.8958779Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-21T01:28:48.9002372Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-21T01:28:48.9021231Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-21T01:28:48.9064157Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-21T01:28:48.9081548Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-21T01:28:48.9124741Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-21T01:28:48.9144756Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-21T01:28:48.9187391Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-21T01:28:48.9205570Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-21T01:28:48.9247125Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-21T01:28:48.9265383Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-21T01:28:48.9308446Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-21T01:28:48.9327536Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-21T01:28:48.9369231Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-21T01:28:48.9390205Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-21T01:28:48.9432355Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-21T01:28:48.9450819Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-21T01:28:48.9492803Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-21T01:28:48.9513189Z Entering 'third_party/mimalloc' 2024-08-21T01:28:48.9558161Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-21T01:28:48.9577919Z Entering 'third_party/nccl/nccl' 2024-08-21T01:28:48.9621257Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-21T01:28:48.9641508Z Entering 'third_party/nlohmann' 2024-08-21T01:28:48.9683713Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-21T01:28:48.9704507Z Entering 'third_party/onnx' 2024-08-21T01:28:48.9746395Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-21T01:28:48.9784568Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-21T01:28:48.9828566Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-21T01:28:48.9848816Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-21T01:28:48.9893187Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-21T01:28:48.9915753Z Entering 'third_party/opentelemetry-cpp' 2024-08-21T01:28:48.9959818Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-21T01:28:48.9981905Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-21T01:28:49.0024928Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-21T01:28:49.0043368Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-21T01:28:49.0085480Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-21T01:28:49.0103666Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-21T01:28:49.0146603Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-21T01:28:49.0165095Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-21T01:28:49.0207823Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-21T01:28:49.0228296Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-21T01:28:49.0271096Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-21T01:28:49.0288941Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-21T01:28:49.0331167Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-21T01:28:49.0350574Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-21T01:28:49.0393412Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-21T01:28:49.0411357Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-21T01:28:49.0453836Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-21T01:28:49.0475523Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-21T01:28:49.0518133Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-21T01:28:49.0540884Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-21T01:28:49.0582236Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-21T01:28:49.0627818Z Entering 'third_party/pocketfft' 2024-08-21T01:28:49.0670167Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-21T01:28:49.0690121Z Entering 'third_party/protobuf' 2024-08-21T01:28:49.0732497Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-21T01:28:49.0756080Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-21T01:28:49.0798700Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-21T01:28:49.0816876Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-21T01:28:49.0859178Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-21T01:28:49.0880921Z Entering 'third_party/psimd' 2024-08-21T01:28:49.0923099Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-21T01:28:49.0942129Z Entering 'third_party/pthreadpool' 2024-08-21T01:28:49.0984649Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-21T01:28:49.1003285Z Entering 'third_party/pybind11' 2024-08-21T01:28:49.1045422Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-21T01:28:49.1065127Z Entering 'third_party/python-peachpy' 2024-08-21T01:28:49.1108389Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-21T01:28:49.1128574Z Entering 'third_party/sleef' 2024-08-21T01:28:49.1171241Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-21T01:28:49.1191887Z Entering 'third_party/tensorpipe' 2024-08-21T01:28:49.1235102Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-21T01:28:49.1253423Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-21T01:28:49.1296308Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-21T01:28:49.1314574Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-21T01:28:49.1358647Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-21T01:28:49.1376256Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-21T01:28:49.1417565Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-21T01:28:49.1437993Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-21T01:28:49.1480820Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-21T01:28:49.1498075Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-21T01:28:49.1542002Z file:/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-21T01:28:49.1939336Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-21T01:28:49.2216155Z Entering 'android/libs/fbjni' 2024-08-21T01:28:49.2258824Z Entering 'third_party/FP16' 2024-08-21T01:28:49.2300480Z Entering 'third_party/FXdiv' 2024-08-21T01:28:49.2341035Z Entering 'third_party/NNPACK' 2024-08-21T01:28:49.2381929Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-21T01:28:49.2423118Z Entering 'third_party/XNNPACK' 2024-08-21T01:28:49.2485625Z Entering 'third_party/benchmark' 2024-08-21T01:28:49.2527114Z Entering 'third_party/cpp-httplib' 2024-08-21T01:28:49.2569025Z Entering 'third_party/cpuinfo' 2024-08-21T01:28:49.2610105Z Entering 'third_party/cudnn_frontend' 2024-08-21T01:28:49.2651044Z Entering 'third_party/cutlass' 2024-08-21T01:28:49.2700310Z Entering 'third_party/eigen' 2024-08-21T01:28:49.2744537Z Entering 'third_party/fbgemm' 2024-08-21T01:28:49.2786984Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-21T01:28:49.2826812Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-21T01:28:49.2865895Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-21T01:28:49.2913256Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-21T01:28:49.2952467Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-21T01:28:49.2992784Z Entering 'third_party/flatbuffers' 2024-08-21T01:28:49.3037113Z Entering 'third_party/fmt' 2024-08-21T01:28:49.3077479Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-21T01:28:49.3119152Z Entering 'third_party/gloo' 2024-08-21T01:28:49.3158816Z Entering 'third_party/googletest' 2024-08-21T01:28:49.3199687Z Entering 'third_party/ideep' 2024-08-21T01:28:49.3237788Z Entering 'third_party/ideep/mkl-dnn' 2024-08-21T01:28:49.3285294Z Entering 'third_party/ittapi' 2024-08-21T01:28:49.3324397Z Entering 'third_party/kineto' 2024-08-21T01:28:49.3364471Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-21T01:28:49.3405742Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-21T01:28:49.3448237Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-21T01:28:49.3487509Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-21T01:28:49.3527944Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-21T01:28:49.3567450Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-21T01:28:49.3608549Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-21T01:28:49.3647781Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-21T01:28:49.3686357Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-21T01:28:49.3726779Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-21T01:28:49.3769553Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-21T01:28:49.3808809Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-21T01:28:49.3850816Z Entering 'third_party/mimalloc' 2024-08-21T01:28:49.3891806Z Entering 'third_party/nccl/nccl' 2024-08-21T01:28:49.3932191Z Entering 'third_party/nlohmann' 2024-08-21T01:28:49.3973687Z Entering 'third_party/onnx' 2024-08-21T01:28:49.4037206Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-21T01:28:49.4078227Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-21T01:28:49.4120732Z Entering 'third_party/opentelemetry-cpp' 2024-08-21T01:28:49.4163263Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-21T01:28:49.4202927Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-21T01:28:49.4242547Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-21T01:28:49.4281240Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-21T01:28:49.4321416Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-21T01:28:49.4361054Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-21T01:28:49.4400044Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-21T01:28:49.4439291Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-21T01:28:49.4481625Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-21T01:28:49.4522288Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-21T01:28:49.4585980Z Entering 'third_party/pocketfft' 2024-08-21T01:28:49.4626229Z Entering 'third_party/protobuf' 2024-08-21T01:28:49.4672027Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-21T01:28:49.4710862Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-21T01:28:49.4751647Z Entering 'third_party/psimd' 2024-08-21T01:28:49.4792131Z Entering 'third_party/pthreadpool' 2024-08-21T01:28:49.4832553Z Entering 'third_party/pybind11' 2024-08-21T01:28:49.4872669Z Entering 'third_party/python-peachpy' 2024-08-21T01:28:49.4913628Z Entering 'third_party/sleef' 2024-08-21T01:28:49.4954062Z Entering 'third_party/tensorpipe' 2024-08-21T01:28:49.4994692Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-21T01:28:49.5034310Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-21T01:28:49.5073618Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-21T01:28:49.5114442Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-21T01:28:49.5152759Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-21T01:28:49.5211616Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-21T01:28:49.5485336Z Entering 'android/libs/fbjni' 2024-08-21T01:28:49.5527049Z Entering 'third_party/FP16' 2024-08-21T01:28:49.5567219Z Entering 'third_party/FXdiv' 2024-08-21T01:28:49.5608180Z Entering 'third_party/NNPACK' 2024-08-21T01:28:49.5649601Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-21T01:28:49.5690621Z Entering 'third_party/XNNPACK' 2024-08-21T01:28:49.5751588Z Entering 'third_party/benchmark' 2024-08-21T01:28:49.5792294Z Entering 'third_party/cpp-httplib' 2024-08-21T01:28:49.5832591Z Entering 'third_party/cpuinfo' 2024-08-21T01:28:49.5875057Z Entering 'third_party/cudnn_frontend' 2024-08-21T01:28:49.5916213Z Entering 'third_party/cutlass' 2024-08-21T01:28:49.5969033Z Entering 'third_party/eigen' 2024-08-21T01:28:49.6013687Z Entering 'third_party/fbgemm' 2024-08-21T01:28:49.6054474Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-21T01:28:49.6093637Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-21T01:28:49.6133165Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-21T01:28:49.6181397Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-21T01:28:49.6220708Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-21T01:28:49.6261301Z Entering 'third_party/flatbuffers' 2024-08-21T01:28:49.6309072Z Entering 'third_party/fmt' 2024-08-21T01:28:49.6350477Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-21T01:28:49.6391522Z Entering 'third_party/gloo' 2024-08-21T01:28:49.6432248Z Entering 'third_party/googletest' 2024-08-21T01:28:49.6473469Z Entering 'third_party/ideep' 2024-08-21T01:28:49.6512937Z Entering 'third_party/ideep/mkl-dnn' 2024-08-21T01:28:49.6561026Z Entering 'third_party/ittapi' 2024-08-21T01:28:49.6604879Z Entering 'third_party/kineto' 2024-08-21T01:28:49.6644902Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-21T01:28:49.6683785Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-21T01:28:49.6726002Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-21T01:28:49.6765914Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-21T01:28:49.6807578Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-21T01:28:49.6848043Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-21T01:28:49.6888340Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-21T01:28:49.6928369Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-21T01:28:49.6969312Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-21T01:28:49.7011360Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-21T01:28:49.7053067Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-21T01:28:49.7093595Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-21T01:28:49.7134941Z Entering 'third_party/mimalloc' 2024-08-21T01:28:49.7179753Z Entering 'third_party/nccl/nccl' 2024-08-21T01:28:49.7220163Z Entering 'third_party/nlohmann' 2024-08-21T01:28:49.7263903Z Entering 'third_party/onnx' 2024-08-21T01:28:49.7325223Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-21T01:28:49.7366031Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-21T01:28:49.7409287Z Entering 'third_party/opentelemetry-cpp' 2024-08-21T01:28:49.7451941Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-21T01:28:49.7491219Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-21T01:28:49.7532226Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-21T01:28:49.7572054Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-21T01:28:49.7612437Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-21T01:28:49.7651206Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-21T01:28:49.7690164Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-21T01:28:49.7729704Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-21T01:28:49.7771829Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-21T01:28:49.7813687Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-21T01:28:49.7876388Z Entering 'third_party/pocketfft' 2024-08-21T01:28:49.7916651Z Entering 'third_party/protobuf' 2024-08-21T01:28:49.7960718Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-21T01:28:49.8000428Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-21T01:28:49.8042048Z Entering 'third_party/psimd' 2024-08-21T01:28:49.8082200Z Entering 'third_party/pthreadpool' 2024-08-21T01:28:49.8122753Z Entering 'third_party/pybind11' 2024-08-21T01:28:49.8163598Z Entering 'third_party/python-peachpy' 2024-08-21T01:28:49.8203589Z Entering 'third_party/sleef' 2024-08-21T01:28:49.8243751Z Entering 'third_party/tensorpipe' 2024-08-21T01:28:49.8284481Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-21T01:28:49.8324403Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-21T01:28:49.8364257Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-21T01:28:49.8404603Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-21T01:28:49.8443038Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-21T01:28:49.8499692Z ##[endgroup] 2024-08-21T01:28:49.8537875Z [command]/usr/bin/git log -1 --format='%H' 2024-08-21T01:28:49.8562506Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-21T01:28:49.8855882Z Prepare all required actions 2024-08-21T01:28:49.8856572Z Getting action download info 2024-08-21T01:28:50.1524152Z ##[group]Run ./.github/actions/setup-linux 2024-08-21T01:28:50.1524676Z env: 2024-08-21T01:28:50.1524992Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:50.1525395Z ##[endgroup] 2024-08-21T01:28:50.1603034Z ##[group]Run set -euo pipefail 2024-08-21T01:28:50.1603568Z set -euo pipefail 2024-08-21T01:28:50.1604033Z function get_ec2_metadata() { 2024-08-21T01:28:50.1604683Z  # Pulled from instance metadata endpoint for EC2 2024-08-21T01:28:50.1605880Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-21T01:28:50.1606866Z  category=$1 2024-08-21T01:28:50.1607608Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-21T01:28:50.1608401Z  runner_name_str=gh-ci-gcp-a100-4 2024-08-21T01:28:50.1609032Z  if [[ -f /.inarc ]]; then 2024-08-21T01:28:50.1609608Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-21T01:28:50.1610295Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-21T01:28:50.1611139Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-21T01:28:50.1611886Z  else 2024-08-21T01:28:50.1612466Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-21T01:28:50.1613163Z  fi 2024-08-21T01:28:50.1613478Z } 2024-08-21T01:28:50.1613898Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-21T01:28:50.1614595Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-21T01:28:50.1615401Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-21T01:28:50.1616140Z echo "system info $(uname -a)" 2024-08-21T01:28:50.1630204Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:50.1630826Z env: 2024-08-21T01:28:50.1631149Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:50.1631554Z ##[endgroup] 2024-08-21T01:28:50.1680710Z ami-id: Runner is from Google Cloud Platform, No info on ec2 metadata 2024-08-21T01:28:50.1686434Z instance-id: Runner is from Google Cloud Platform, No info on ec2 metadata 2024-08-21T01:28:50.1692802Z instance-type: Runner is from Google Cloud Platform, No info on ec2 metadata 2024-08-21T01:28:50.1704728Z system info Linux gh-ci-gcp-a100-4 5.15.0-1062-gcp #70~20.04.1-Ubuntu SMP Fri May 24 20:12:18 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-21T01:28:50.1731075Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-21T01:28:50.1732247Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-21T01:28:50.1745265Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:50.1745923Z env: 2024-08-21T01:28:50.1746443Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:50.1746855Z ##[endgroup] 2024-08-21T01:28:50.1837043Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-21T01:28:50.1837783Z if systemctl is-active --quiet docker; then 2024-08-21T01:28:50.1838450Z  echo "Docker daemon is running..."; 2024-08-21T01:28:50.1838993Z else 2024-08-21T01:28:50.1839573Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-21T01:28:50.1840288Z fi 2024-08-21T01:28:50.1851930Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:50.1852542Z env: 2024-08-21T01:28:50.1852846Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:50.1853252Z ##[endgroup] 2024-08-21T01:28:50.1923558Z Docker daemon is running... 2024-08-21T01:28:50.1989881Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-21T01:28:50.1990579Z with: 2024-08-21T01:28:50.1990890Z shell: bash 2024-08-21T01:28:50.1991254Z timeout_minutes: 5 2024-08-21T01:28:50.1991634Z max_attempts: 3 2024-08-21T01:28:50.1992274Z retry_wait_seconds: 30 2024-08-21T01:28:50.1996285Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-21T01:28:50.2000405Z polling_interval_seconds: 1 2024-08-21T01:28:50.2000857Z warning_on_retry: true 2024-08-21T01:28:50.2001265Z continue_on_error: false 2024-08-21T01:28:50.2001661Z env: 2024-08-21T01:28:50.2001952Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:50.2002412Z AWS_RETRY_MODE: standard 2024-08-21T01:28:50.2002833Z AWS_MAX_ATTEMPTS: 5 2024-08-21T01:28:50.2003218Z AWS_DEFAULT_REGION: us-east-1 2024-08-21T01:28:50.2003658Z ##[endgroup] 2024-08-21T01:28:52.4714831Z WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json. 2024-08-21T01:28:52.4716122Z Configure a credential helper to remove this warning. See 2024-08-21T01:28:52.4717447Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-21T01:28:52.4718162Z 2024-08-21T01:28:52.4718296Z Login Succeeded 2024-08-21T01:28:53.2576259Z Command completed after 1 attempt(s). 2024-08-21T01:28:53.2648485Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-21T01:28:53.2649403Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-21T01:28:53.2650200Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-21T01:28:53.2663763Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:53.2664401Z env: 2024-08-21T01:28:53.2664737Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:53.2665134Z ##[endgroup] 2024-08-21T01:28:53.2766435Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-21T01:28:53.2767416Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-21T01:28:53.2768148Z # shellcheck disable=SC2046 2024-08-21T01:28:53.2768706Z docker stop $(docker ps -q) || true 2024-08-21T01:28:53.2769289Z # Prune all of the docker images 2024-08-21T01:28:53.2769819Z docker system prune -af 2024-08-21T01:28:53.2782427Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:53.2783035Z env: 2024-08-21T01:28:53.2783354Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:53.2783749Z ##[endgroup] 2024-08-21T01:28:53.3229975Z "docker stop" requires at least 1 argument. 2024-08-21T01:28:53.3231329Z See 'docker stop --help'. 2024-08-21T01:28:53.3232090Z 2024-08-21T01:28:53.3232364Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-21T01:28:53.3232847Z 2024-08-21T01:28:53.3233032Z Stop one or more running containers 2024-08-21T01:28:53.3470605Z Total reclaimed space: 0B 2024-08-21T01:28:53.3513309Z ##[group]Run set +e 2024-08-21T01:28:53.3513750Z set +e 2024-08-21T01:28:53.3514082Z set -x 2024-08-21T01:28:53.3514423Z  2024-08-21T01:28:53.3514800Z PT_DOMAIN=download.pytorch.org 2024-08-21T01:28:53.3515806Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-21T01:28:53.3517172Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-21T01:28:53.3518161Z # one is returned at random 2024-08-21T01:28:53.3518818Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-21T01:28:53.3519439Z  2024-08-21T01:28:53.3519790Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-21T01:28:53.3520792Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-21T01:28:53.3521749Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-21T01:28:53.3522433Z  2024-08-21T01:28:53.3522789Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-21T01:28:53.3523457Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-21T01:28:53.3524073Z  exit 1 2024-08-21T01:28:53.3524436Z  fi 2024-08-21T01:28:53.3524746Z fi 2024-08-21T01:28:53.3525056Z  2024-08-21T01:28:53.3525470Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-21T01:28:53.3526098Z  # Clean up any old records first 2024-08-21T01:28:53.3526696Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-21T01:28:53.3527244Z fi 2024-08-21T01:28:53.3527551Z  2024-08-21T01:28:53.3528056Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-21T01:28:53.3528756Z cat /etc/hosts 2024-08-21T01:28:53.3542052Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:53.3542678Z env: 2024-08-21T01:28:53.3543004Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:53.3543400Z ##[endgroup] 2024-08-21T01:28:53.3579336Z + PT_DOMAIN=download.pytorch.org 2024-08-21T01:28:53.3586667Z ++ dig -4 +short download.pytorch.org 2024-08-21T01:28:53.3587290Z ++ tail -n1 2024-08-21T01:28:53.3783588Z + RESOLVED_IP=13.249.85.22 2024-08-21T01:28:53.3785043Z + '[' -z 13.249.85.22 ']' 2024-08-21T01:28:53.3785718Z + grep -r download.pytorch.org /etc/hosts 2024-08-21T01:28:53.3797238Z 13.249.85.22 download.pytorch.org 2024-08-21T01:28:53.3798877Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-21T01:28:53.3912609Z + echo '13.249.85.22 download.pytorch.org' 2024-08-21T01:28:53.3913252Z + sudo tee -a /etc/hosts 2024-08-21T01:28:53.3985705Z 13.249.85.22 download.pytorch.org 2024-08-21T01:28:53.3993210Z + cat /etc/hosts 2024-08-21T01:28:53.4001481Z 127.0.0.1 localhost 2024-08-21T01:28:53.4001798Z 2024-08-21T01:28:53.4002125Z # The following lines are desirable for IPv6 capable hosts 2024-08-21T01:28:53.4016891Z ::1 ip6-localhost ip6-loopback 2024-08-21T01:28:53.4017468Z fe00::0 ip6-localnet 2024-08-21T01:28:53.4017905Z ff00::0 ip6-mcastprefix 2024-08-21T01:28:53.4018364Z ff02::1 ip6-allnodes 2024-08-21T01:28:53.4018785Z ff02::2 ip6-allrouters 2024-08-21T01:28:53.4019195Z ff02::3 ip6-allhosts 2024-08-21T01:28:53.4019668Z 169.254.169.254 metadata.google.internal metadata 2024-08-21T01:28:53.4020262Z 13.249.85.22 download.pytorch.org 2024-08-21T01:28:53.4222903Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-21T01:28:53.4223655Z with: 2024-08-21T01:28:53.4225111Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:53.4226801Z docker-build-dir: .ci/docker 2024-08-21T01:28:53.4227745Z working-directory: . 2024-08-21T01:28:53.4228489Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:53.4229146Z force-push: false 2024-08-21T01:28:53.4229486Z env: 2024-08-21T01:28:53.4229794Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:53.4230204Z ##[endgroup] 2024-08-21T01:28:53.4252794Z ##[group]Run set -ex 2024-08-21T01:28:53.4253225Z set -ex 2024-08-21T01:28:53.4253560Z  2024-08-21T01:28:53.4254227Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-21T01:28:53.4255500Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-21T01:28:53.4256562Z # job could then download the pre-built image as usual 2024-08-21T01:28:53.4257485Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-21T01:28:53.4258342Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-21T01:28:53.4259144Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-21T01:28:53.4259823Z  2024-08-21T01:28:53.4260449Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-21T01:28:53.4261224Z  exit 0 2024-08-21T01:28:53.4261568Z else 2024-08-21T01:28:53.4261981Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-21T01:28:53.4262524Z fi 2024-08-21T01:28:53.4262833Z  2024-08-21T01:28:53.4263397Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-21T01:28:53.4264463Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-21T01:28:53.4265440Z  # use it as it is, but first let's extract the tag 2024-08-21T01:28:53.4266346Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-21T01:28:53.4267242Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-21T01:28:53.4268433Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-21T01:28:53.4269126Z else 2024-08-21T01:28:53.4269649Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-21T01:28:53.4270470Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-21T01:28:53.4271610Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-21T01:28:53.4272549Z fi 2024-08-21T01:28:53.4285523Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:53.4286187Z env: 2024-08-21T01:28:53.4286504Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:53.4286908Z REPO_NAME: pytorch 2024-08-21T01:28:53.4288411Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:53.4290043Z DOCKER_BUILD_DIR: .ci/docker 2024-08-21T01:28:53.4290653Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:53.4291291Z ##[endgroup] 2024-08-21T01:28:53.4327374Z + [[ ! -d .ci/docker ]] 2024-08-21T01:28:53.4327903Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-21T01:28:53.4328348Z + echo skip=false 2024-08-21T01:28:53.4331049Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-21T01:28:53.4336970Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:53.4338758Z ++ awk -F '[:,]' '{print $2}' 2024-08-21T01:28:53.4360044Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:53.4360997Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:53.4364251Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:53.4397419Z ##[group]Run set +e 2024-08-21T01:28:53.4397853Z set +e 2024-08-21T01:28:53.4398203Z set -x 2024-08-21T01:28:53.4398542Z  2024-08-21T01:28:53.4398839Z login() { 2024-08-21T01:28:53.4399666Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-21T01:28:53.4400570Z } 2024-08-21T01:28:53.4400882Z  2024-08-21T01:28:53.4401182Z retry () { 2024-08-21T01:28:53.4401639Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-21T01:28:53.4402178Z } 2024-08-21T01:28:53.4402489Z  2024-08-21T01:28:53.4402837Z retry login "${DOCKER_REGISTRY}" 2024-08-21T01:28:53.4403346Z  2024-08-21T01:28:53.4403915Z # Check if image already exists, if it does then skip building it 2024-08-21T01:28:53.4404805Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-21T01:28:53.4405404Z  exit 0 2024-08-21T01:28:53.4405797Z fi 2024-08-21T01:28:53.4406112Z  2024-08-21T01:28:53.4406719Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-21T01:28:53.4407752Z # be empty. The default action would be to continue rebuild the image 2024-08-21T01:28:53.4408683Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-21T01:28:53.4409523Z  # if we're on the base branch then use the parent commit 2024-08-21T01:28:53.4410250Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-21T01:28:53.4410754Z else 2024-08-21T01:28:53.4411322Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-21T01:28:53.4412189Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-21T01:28:53.4412806Z fi 2024-08-21T01:28:53.4413107Z  2024-08-21T01:28:53.4413473Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-21T01:28:53.4414082Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-21T01:28:53.4414636Z  2024-08-21T01:28:53.4415438Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-21T01:28:53.4416436Z  exit 0 2024-08-21T01:28:53.4416781Z fi 2024-08-21T01:28:53.4417096Z  2024-08-21T01:28:53.4417606Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-21T01:28:53.4418844Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-21T01:28:53.4419860Z  exit 1 2024-08-21T01:28:53.4420202Z fi 2024-08-21T01:28:53.4420498Z  2024-08-21T01:28:53.4421100Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-21T01:28:53.4422311Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-21T01:28:53.4423410Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-21T01:28:53.4424623Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-21T01:28:53.4426066Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-21T01:28:53.4427036Z fi 2024-08-21T01:28:53.4427366Z  2024-08-21T01:28:53.4427924Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-21T01:28:53.4440367Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:53.4440978Z env: 2024-08-21T01:28:53.4441293Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:53.4441726Z DOCKER_BUILD_DIR: .ci/docker 2024-08-21T01:28:53.4442448Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:28:53.4444134Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:53.4445850Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:53.4446603Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:53.4447224Z ##[endgroup] 2024-08-21T01:28:53.4482229Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:53.4483048Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:53.4485119Z + aws ecr get-login-password --region us-east-1 2024-08-21T01:28:53.4486762Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:54.6181514Z WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json. 2024-08-21T01:28:54.6183159Z Configure a credential helper to remove this warning. See 2024-08-21T01:28:54.6184731Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-21T01:28:54.6185487Z 2024-08-21T01:28:54.6185623Z Login Succeeded 2024-08-21T01:28:54.6207896Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:55.5543654Z { 2024-08-21T01:28:55.5544082Z "schemaVersion": 2, 2024-08-21T01:28:55.5545000Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-21T01:28:55.5546093Z "config": { 2024-08-21T01:28:55.5546955Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-21T01:28:55.5548399Z "size": 48711, 2024-08-21T01:28:55.5549862Z "digest": "sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb" 2024-08-21T01:28:55.5552052Z }, 2024-08-21T01:28:55.5552386Z "layers": [ 2024-08-21T01:28:55.5552751Z { 2024-08-21T01:28:55.5553300Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5553992Z "size": 28580681, 2024-08-21T01:28:55.5554726Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-21T01:28:55.5555569Z }, 2024-08-21T01:28:55.5555894Z { 2024-08-21T01:28:55.5556410Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5557104Z "size": 7943451, 2024-08-21T01:28:55.5557775Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-08-21T01:28:55.5558555Z }, 2024-08-21T01:28:55.5558819Z { 2024-08-21T01:28:55.5559348Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5560071Z "size": 55728572, 2024-08-21T01:28:55.5560747Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-08-21T01:28:55.5561498Z }, 2024-08-21T01:28:55.5561774Z { 2024-08-21T01:28:55.5562304Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5562983Z "size": 186, 2024-08-21T01:28:55.5563666Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-08-21T01:28:55.5564449Z }, 2024-08-21T01:28:55.5564722Z { 2024-08-21T01:28:55.5565228Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5565974Z "size": 6886, 2024-08-21T01:28:55.5566648Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-08-21T01:28:55.5567431Z }, 2024-08-21T01:28:55.5568114Z { 2024-08-21T01:28:55.5568724Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5569685Z "size": 1329001756, 2024-08-21T01:28:55.5570592Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-08-21T01:28:55.5571374Z }, 2024-08-21T01:28:55.5571652Z { 2024-08-21T01:28:55.5572179Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5573086Z "size": 62501, 2024-08-21T01:28:55.5573766Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-08-21T01:28:55.5574549Z }, 2024-08-21T01:28:55.5574827Z { 2024-08-21T01:28:55.5575338Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5576074Z "size": 1684, 2024-08-21T01:28:55.5576737Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-08-21T01:28:55.5577502Z }, 2024-08-21T01:28:55.5577766Z { 2024-08-21T01:28:55.5578320Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5579010Z "size": 1523, 2024-08-21T01:28:55.5579698Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-08-21T01:28:55.5580501Z }, 2024-08-21T01:28:55.5580776Z { 2024-08-21T01:28:55.5581303Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5581985Z "size": 2528295403, 2024-08-21T01:28:55.5582685Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-08-21T01:28:55.5583460Z }, 2024-08-21T01:28:55.5583738Z { 2024-08-21T01:28:55.5584245Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5584944Z "size": 86016, 2024-08-21T01:28:55.5585678Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-08-21T01:28:55.5586464Z }, 2024-08-21T01:28:55.5586729Z { 2024-08-21T01:28:55.5587260Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5588256Z "size": 1822, 2024-08-21T01:28:55.5588926Z "digest": "sha256:ac23c5fcc1ef310e9cd0d138b180281188e20a3e28c51cd14751622e3c7d92ea" 2024-08-21T01:28:55.5589714Z }, 2024-08-21T01:28:55.5589995Z { 2024-08-21T01:28:55.5590520Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5591199Z "size": 246761563, 2024-08-21T01:28:55.5591921Z "digest": "sha256:ca426ffc9d2bd051ad1ff5c4f02fd5d9be161e70953cc37882c815080c708f4d" 2024-08-21T01:28:55.5592711Z }, 2024-08-21T01:28:55.5592984Z { 2024-08-21T01:28:55.5593494Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5594180Z "size": 544, 2024-08-21T01:28:55.5594826Z "digest": "sha256:4373aa743caf6660c671295b837048c8350a2165f2336dc3b7b6c96462579e50" 2024-08-21T01:28:55.5595569Z }, 2024-08-21T01:28:55.5595887Z { 2024-08-21T01:28:55.5596416Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5597116Z "size": 1258, 2024-08-21T01:28:55.5597763Z "digest": "sha256:75b2abb16c685854c017d0cf75c626526c956734c656d9e78fdd4520317e9b3c" 2024-08-21T01:28:55.5598529Z }, 2024-08-21T01:28:55.5598806Z { 2024-08-21T01:28:55.5599326Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5600000Z "size": 484, 2024-08-21T01:28:55.5600647Z "digest": "sha256:ef1e3ef1e85f0124270b8506721219150e230184181f8207d64463bf5c67e656" 2024-08-21T01:28:55.5601396Z }, 2024-08-21T01:28:55.5601667Z { 2024-08-21T01:28:55.5602173Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5602857Z "size": 91711766, 2024-08-21T01:28:55.5603537Z "digest": "sha256:0bf61114d37b2d09065aa6852b1e78a32580a1ff011dd5d23f03b0f5c4e710af" 2024-08-21T01:28:55.5604293Z }, 2024-08-21T01:28:55.5604574Z { 2024-08-21T01:28:55.5605095Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5605815Z "size": 3390, 2024-08-21T01:28:55.5606685Z "digest": "sha256:2df849f72ba4258df10c9c45ace21a66d93b73196e78e96f12c13098822f60e2" 2024-08-21T01:28:55.5607476Z }, 2024-08-21T01:28:55.5607753Z { 2024-08-21T01:28:55.5608284Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5608962Z "size": 1909, 2024-08-21T01:28:55.5609619Z "digest": "sha256:34c47d63c5de27418739aa048b4dd781c3006d65691e9b903c95304d0509d89c" 2024-08-21T01:28:55.5610520Z }, 2024-08-21T01:28:55.5610783Z { 2024-08-21T01:28:55.5611315Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5612000Z "size": 701, 2024-08-21T01:28:55.5612670Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-21T01:28:55.5613434Z }, 2024-08-21T01:28:55.5613709Z { 2024-08-21T01:28:55.5614228Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5614917Z "size": 2864254879, 2024-08-21T01:28:55.5615652Z "digest": "sha256:6efb96fc87ea1bca54876d85331e5585e28f325a1d60ee1aef7811eabb74f933" 2024-08-21T01:28:55.5616438Z }, 2024-08-21T01:28:55.5616714Z { 2024-08-21T01:28:55.5617239Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5617904Z "size": 380, 2024-08-21T01:28:55.5618561Z "digest": "sha256:b119c4fc6c58d9e72744201ff4850b34184a94eb3566031b7d36e87ecce950a6" 2024-08-21T01:28:55.5619328Z }, 2024-08-21T01:28:55.5619588Z { 2024-08-21T01:28:55.5620113Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5620796Z "size": 12900, 2024-08-21T01:28:55.5621492Z "digest": "sha256:33b639226eeefa41d7bb26a8ecc20d271ceef45571dabfd80aaee3634926381b" 2024-08-21T01:28:55.5622260Z }, 2024-08-21T01:28:55.5622534Z { 2024-08-21T01:28:55.5623055Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5623744Z "size": 803, 2024-08-21T01:28:55.5624384Z "digest": "sha256:d3547bd43146930aa830d54562c98aa678e355cf5f84527e9717eb72fffe8ca3" 2024-08-21T01:28:55.5625148Z }, 2024-08-21T01:28:55.5625420Z { 2024-08-21T01:28:55.5625959Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5626649Z "size": 106, 2024-08-21T01:28:55.5627306Z "digest": "sha256:5645520256fa6d7f2270608eaed53849a131d03fbf4c5d4472c9cfde1a439e47" 2024-08-21T01:28:55.5628377Z }, 2024-08-21T01:28:55.5628654Z { 2024-08-21T01:28:55.5629184Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5629869Z "size": 503, 2024-08-21T01:28:55.5630530Z "digest": "sha256:829ed7d91e3818d0e11834c71551117c7aeec0a51d798e93d641faa0c02582cb" 2024-08-21T01:28:55.5631281Z }, 2024-08-21T01:28:55.5631557Z { 2024-08-21T01:28:55.5632077Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5632769Z "size": 121477267, 2024-08-21T01:28:55.5633436Z "digest": "sha256:385d0421d67c533d628985c453a186028a61f6ad4209ae519ec3d4ecea661d3e" 2024-08-21T01:28:55.5634198Z }, 2024-08-21T01:28:55.5634473Z { 2024-08-21T01:28:55.5634984Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5635722Z "size": 109, 2024-08-21T01:28:55.5636392Z "digest": "sha256:7cbfce999d76851e07dd71056a6d852629d789ccfd46be44cc405b451fb8bb58" 2024-08-21T01:28:55.5637171Z }, 2024-08-21T01:28:55.5637434Z { 2024-08-21T01:28:55.5637960Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5638648Z "size": 490, 2024-08-21T01:28:55.5639306Z "digest": "sha256:6658e558ebe496f0d628947534b0ea1e44808f84f2dfd8a9a704e8aa5329d791" 2024-08-21T01:28:55.5640055Z }, 2024-08-21T01:28:55.5640332Z { 2024-08-21T01:28:55.5640858Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5641529Z "size": 297, 2024-08-21T01:28:55.5642186Z "digest": "sha256:33444a5a7e01a3010fd1cc16b32a66892a08600d7f700747d23faf3c36624f3a" 2024-08-21T01:28:55.5642946Z }, 2024-08-21T01:28:55.5643434Z { 2024-08-21T01:28:55.5643953Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5644647Z "size": 103, 2024-08-21T01:28:55.5645322Z "digest": "sha256:7c47122f4e09b47e891b4ae9e499a35a4eb46faacae87d0e946615adda6e0706" 2024-08-21T01:28:55.5646137Z }, 2024-08-21T01:28:55.5646401Z { 2024-08-21T01:28:55.5646926Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5647801Z "size": 1473, 2024-08-21T01:28:55.5648481Z "digest": "sha256:1811ff4d3556ac255604f346d879528aff2aadd7652c2d39baf4fe6eaa60aec3" 2024-08-21T01:28:55.5649236Z }, 2024-08-21T01:28:55.5649517Z { 2024-08-21T01:28:55.5650071Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5650746Z "size": 594729516, 2024-08-21T01:28:55.5651429Z "digest": "sha256:7754ee068b33f760d9c320c26a1a3935247ace0c8c463a976105c1586a43fb1c" 2024-08-21T01:28:55.5652179Z }, 2024-08-21T01:28:55.5652453Z { 2024-08-21T01:28:55.5652965Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5653651Z "size": 164, 2024-08-21T01:28:55.5654318Z "digest": "sha256:0479fbac3361713c075bf788e778117917dcacdd1ff71ed79a789dd2016a1714" 2024-08-21T01:28:55.5655090Z }, 2024-08-21T01:28:55.5655351Z { 2024-08-21T01:28:55.5655908Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5656599Z "size": 564, 2024-08-21T01:28:55.5657241Z "digest": "sha256:f1d6874dee9c34d154278d2b6935d04447f07fa89d25d4a09398234040d8a7e8" 2024-08-21T01:28:55.5658048Z }, 2024-08-21T01:28:55.5658325Z { 2024-08-21T01:28:55.5658852Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5659530Z "size": 35874203, 2024-08-21T01:28:55.5660267Z "digest": "sha256:afc429e87b593a64f3e0a1be51e09e966ea1aa71e6e8a0bec4edc9f2f7bc1c05" 2024-08-21T01:28:55.5661059Z }, 2024-08-21T01:28:55.5661337Z { 2024-08-21T01:28:55.5661852Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5662538Z "size": 104, 2024-08-21T01:28:55.5663191Z "digest": "sha256:019620471c2592cd22437fe8ed1a94c784114cf7445c5c2ab36dd0bae727fc66" 2024-08-21T01:28:55.5663946Z }, 2024-08-21T01:28:55.5664209Z { 2024-08-21T01:28:55.5664733Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5665416Z "size": 425, 2024-08-21T01:28:55.5666139Z "digest": "sha256:27aeb1b2cd5e28d8f0e54fc1090baf88abc29ebf5fce088b493fcae99c9550d1" 2024-08-21T01:28:55.5666940Z }, 2024-08-21T01:28:55.5667220Z { 2024-08-21T01:28:55.5668046Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5668729Z "size": 20262183, 2024-08-21T01:28:55.5669416Z "digest": "sha256:829c0e19f4b7797e9120a86172fabfd710f9b3971ffd714a616931a3fadc7192" 2024-08-21T01:28:55.5670184Z }, 2024-08-21T01:28:55.5693411Z { 2024-08-21T01:28:55.5694100Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5694837Z "size": 438, 2024-08-21T01:28:55.5695515Z "digest": "sha256:560cd6122a3868e7e82e75eb103d182edf6512114d61ccdfae099af00cb16b4c" 2024-08-21T01:28:55.5696354Z }, 2024-08-21T01:28:55.5696633Z { 2024-08-21T01:28:55.5697158Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5697849Z "size": 701, 2024-08-21T01:28:55.5698510Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-21T01:28:55.5699306Z }, 2024-08-21T01:28:55.5699584Z { 2024-08-21T01:28:55.5700092Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5700778Z "size": 141, 2024-08-21T01:28:55.5701452Z "digest": "sha256:5a4eb19d72198a01f3babd4711c40fcf7c6e5cb9d5cfedc507b0061408d47097" 2024-08-21T01:28:55.5702224Z }, 2024-08-21T01:28:55.5702482Z { 2024-08-21T01:28:55.5702997Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5703685Z "size": 135, 2024-08-21T01:28:55.5704588Z "digest": "sha256:da6a838a66c72b5de9a5590c233779880ea901c1264940a1c618825d2501ffd1" 2024-08-21T01:28:55.5705346Z }, 2024-08-21T01:28:55.5705670Z { 2024-08-21T01:28:55.5706193Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5706887Z "size": 4838068375, 2024-08-21T01:28:55.5707943Z "digest": "sha256:cbb79a7053537bed2d80b77198ea552b985563b11d3f8616593c6ab575a3a090" 2024-08-21T01:28:55.5708982Z }, 2024-08-21T01:28:55.5709248Z { 2024-08-21T01:28:55.5709757Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5710446Z "size": 194, 2024-08-21T01:28:55.5711116Z "digest": "sha256:4d4fa6802ab3e6b36d8d4adb650046e9370c4a16a19876970559ef8a312ca855" 2024-08-21T01:28:55.5711897Z }, 2024-08-21T01:28:55.5712155Z { 2024-08-21T01:28:55.5712677Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5713367Z "size": 564, 2024-08-21T01:28:55.5714043Z "digest": "sha256:0f085d4fa279366fcd52d303b7ad8abddc5962a8872174b8109f3f5d5ca5f2cf" 2024-08-21T01:28:55.5714807Z }, 2024-08-21T01:28:55.5715085Z { 2024-08-21T01:28:55.5715641Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5716315Z "size": 43162921, 2024-08-21T01:28:55.5716997Z "digest": "sha256:4a0412373305517489d891a4749e6d2b80b4aeb7df3b8adca0ed0abb2f3e5300" 2024-08-21T01:28:55.5717769Z }, 2024-08-21T01:28:55.5718047Z { 2024-08-21T01:28:55.5718553Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5719237Z "size": 106, 2024-08-21T01:28:55.5719891Z "digest": "sha256:302c8c1779444e54919f83e91c6d5e1fdea653fc899be802b3a2a34d34d99d5d" 2024-08-21T01:28:55.5720663Z }, 2024-08-21T01:28:55.5720922Z { 2024-08-21T01:28:55.5721438Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5722123Z "size": 1380, 2024-08-21T01:28:55.5722774Z "digest": "sha256:02970347dff622ee45629bc28788716d00528464f71f88ec7b74ae5f44a486e2" 2024-08-21T01:28:55.5723510Z }, 2024-08-21T01:28:55.5723783Z { 2024-08-21T01:28:55.5724299Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5724973Z "size": 701, 2024-08-21T01:28:55.5725718Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-21T01:28:55.5726498Z }, 2024-08-21T01:28:55.5726778Z { 2024-08-21T01:28:55.5727280Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5727963Z "size": 139, 2024-08-21T01:28:55.5728616Z "digest": "sha256:916c5e8580164926a3aec87e88a5d1c2d02756e49fdeed4198e1e6304efa2fd3" 2024-08-21T01:28:55.5729382Z }, 2024-08-21T01:28:55.5729643Z { 2024-08-21T01:28:55.5730163Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5730848Z "size": 118, 2024-08-21T01:28:55.5731510Z "digest": "sha256:c42b3602a1fb4df5111224650f15adffb025979ae9c614807379ea84e189533b" 2024-08-21T01:28:55.5732261Z }, 2024-08-21T01:28:55.5732536Z { 2024-08-21T01:28:55.5733053Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5733725Z "size": 4324672728, 2024-08-21T01:28:55.5734439Z "digest": "sha256:11fad972ef6cb9d2dc236504b0ee5b6e5ea996e4576ece0ed3262fbb9ee2de60" 2024-08-21T01:28:55.5735225Z }, 2024-08-21T01:28:55.5735496Z { 2024-08-21T01:28:55.5736036Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5736800Z "size": 174, 2024-08-21T01:28:55.5737453Z "digest": "sha256:f957c16244819132b478272aaf24231699c4fcbd495c229a574f94daab8d6b86" 2024-08-21T01:28:55.5738345Z }, 2024-08-21T01:28:55.5738609Z { 2024-08-21T01:28:55.5739134Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5739824Z "size": 907, 2024-08-21T01:28:55.5740495Z "digest": "sha256:768c31dca016b80bbdc973c8ee818034bdbd17b7ea9bdadbf0ec45ea51005720" 2024-08-21T01:28:55.5741289Z }, 2024-08-21T01:28:55.5741747Z { 2024-08-21T01:28:55.5742283Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5742960Z "size": 701, 2024-08-21T01:28:55.5743636Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-21T01:28:55.5744412Z }, 2024-08-21T01:28:55.5744685Z { 2024-08-21T01:28:55.5745186Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5746023Z "size": 134, 2024-08-21T01:28:55.5746701Z "digest": "sha256:8c32d0ecebbe2054c96a3a5c1f3d6c8b87c2e826686b369e5242cd55d0f01c2d" 2024-08-21T01:28:55.5747663Z }, 2024-08-21T01:28:55.5748052Z { 2024-08-21T01:28:55.5748583Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5749270Z "size": 32, 2024-08-21T01:28:55.5749959Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-21T01:28:55.5750741Z }, 2024-08-21T01:28:55.5751016Z { 2024-08-21T01:28:55.5751548Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5752219Z "size": 158, 2024-08-21T01:28:55.5752891Z "digest": "sha256:78ee77eddef23f20afc78e2924f4467041cadb754634ee3df431c68211cf676d" 2024-08-21T01:28:55.5753669Z }, 2024-08-21T01:28:55.5753939Z { 2024-08-21T01:28:55.5754446Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5755141Z "size": 1841, 2024-08-21T01:28:55.5755856Z "digest": "sha256:37513e9b43d7123e7ddee09100f62a72b57af7b3833eaecd515e247e2f63e13f" 2024-08-21T01:28:55.5756612Z }, 2024-08-21T01:28:55.5756886Z { 2024-08-21T01:28:55.5757405Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5758101Z "size": 7529776, 2024-08-21T01:28:55.5758771Z "digest": "sha256:a7a5392e2bf6b73d9be292cc22a99ad4637636c15733beb7afbe6403b5ba21a8" 2024-08-21T01:28:55.5759548Z }, 2024-08-21T01:28:55.5759846Z { 2024-08-21T01:28:55.5760372Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5761041Z "size": 163, 2024-08-21T01:28:55.5761697Z "digest": "sha256:b10de02255c021b2366b85bf12ddf593aac89f88964201c455b71a145bc5ca2e" 2024-08-21T01:28:55.5762464Z }, 2024-08-21T01:28:55.5762738Z { 2024-08-21T01:28:55.5763241Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5763925Z "size": 7943, 2024-08-21T01:28:55.5764607Z "digest": "sha256:e56eeff2db93ffe5002a2bb764795b7333321c6802cc28bef0801a619fe6a07e" 2024-08-21T01:28:55.5765375Z }, 2024-08-21T01:28:55.5765687Z { 2024-08-21T01:28:55.5766206Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5766893Z "size": 8068, 2024-08-21T01:28:55.5767556Z "digest": "sha256:70ac3e38d0fd1f0f46ad710c3fc485fd01406606054e8cf84a63af1570bc9f3a" 2024-08-21T01:28:55.5768334Z }, 2024-08-21T01:28:55.5768607Z { 2024-08-21T01:28:55.5769127Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5769807Z "size": 302, 2024-08-21T01:28:55.5770463Z "digest": "sha256:eb09cbf3e4168749cc8605e3188ed35070f6037d13930e443c38d8941734488f" 2024-08-21T01:28:55.5771226Z }, 2024-08-21T01:28:55.5771484Z { 2024-08-21T01:28:55.5772006Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5772688Z "size": 7629638, 2024-08-21T01:28:55.5773352Z "digest": "sha256:ee1273153675e6543b58f5be50321abee5e91f7351701d5d501a6837eaf676ef" 2024-08-21T01:28:55.5774102Z }, 2024-08-21T01:28:55.5774375Z { 2024-08-21T01:28:55.5774891Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5775573Z "size": 108, 2024-08-21T01:28:55.5776263Z "digest": "sha256:e22979f9e0eedfa16f98eb4fa26f30a50c237f5ffd2f6e5b898f2b908058496d" 2024-08-21T01:28:55.5777118Z + exit 0 2024-08-21T01:28:55.5777410Z }, 2024-08-21T01:28:55.5777686Z { 2024-08-21T01:28:55.5778189Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5779094Z "size": 54145661, 2024-08-21T01:28:55.5779817Z "digest": "sha256:6c1f0aabce7c84f9b0b9ab3b96325778dba1c1465b7463c5b4c6be1cf52b74db" 2024-08-21T01:28:55.5780603Z }, 2024-08-21T01:28:55.5780862Z { 2024-08-21T01:28:55.5781382Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5782071Z "size": 473, 2024-08-21T01:28:55.5782744Z "digest": "sha256:7ebe49c3a23e71b289ce012b48dad35791a2f2aa7fcee2326a64d763b0d14df8" 2024-08-21T01:28:55.5783629Z }, 2024-08-21T01:28:55.5783906Z { 2024-08-21T01:28:55.5784427Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5785105Z "size": 1374858766, 2024-08-21T01:28:55.5785831Z "digest": "sha256:5d115b29b5d53e5418762bc41ae975dd245bcffc3d058b8902c58146d505cb37" 2024-08-21T01:28:55.5786596Z }, 2024-08-21T01:28:55.5786867Z { 2024-08-21T01:28:55.5787371Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5788414Z "size": 106, 2024-08-21T01:28:55.5789110Z "digest": "sha256:7f2d7a04fbe65bbe1e895138e0cfeeef53ad926c890bab8e7e132ad1e655e48f" 2024-08-21T01:28:55.5789906Z }, 2024-08-21T01:28:55.5790166Z { 2024-08-21T01:28:55.5790685Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5791373Z "size": 559, 2024-08-21T01:28:55.5792015Z "digest": "sha256:d79fd7d526989e554455eda7d459f2c88444030f61710101e52ec8c7a49b7b79" 2024-08-21T01:28:55.5792778Z }, 2024-08-21T01:28:55.5793053Z { 2024-08-21T01:28:55.5793570Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5794245Z "size": 46248488, 2024-08-21T01:28:55.5794935Z "digest": "sha256:9752d6fbbcab4e7403f26777c5b65911320de6389c81b833a47ba45c4f3354b4" 2024-08-21T01:28:55.5795744Z }, 2024-08-21T01:28:55.5796020Z { 2024-08-21T01:28:55.5796529Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5797220Z "size": 111, 2024-08-21T01:28:55.5797895Z "digest": "sha256:83808b2eff4d71ed46db7c29f66fe918f328405c25b314225dfad01b4868d1f4" 2024-08-21T01:28:55.5798668Z }, 2024-08-21T01:28:55.5798930Z { 2024-08-21T01:28:55.5799449Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5800132Z "size": 32, 2024-08-21T01:28:55.5800787Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-21T01:28:55.5801567Z }, 2024-08-21T01:28:55.5801843Z { 2024-08-21T01:28:55.5802363Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5803029Z "size": 32, 2024-08-21T01:28:55.5803692Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-21T01:28:55.5804468Z }, 2024-08-21T01:28:55.5804740Z { 2024-08-21T01:28:55.5805238Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5805958Z "size": 32, 2024-08-21T01:28:55.5806629Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-21T01:28:55.5807396Z }, 2024-08-21T01:28:55.5807672Z { 2024-08-21T01:28:55.5808195Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-21T01:28:55.5808876Z "size": 32, 2024-08-21T01:28:55.5809526Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-21T01:28:55.5810301Z } 2024-08-21T01:28:55.5810567Z ] 2024-08-21T01:28:55.5810843Z } 2024-08-21T01:28:55.5944121Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-21T01:28:55.5944728Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-21T01:28:55.5945397Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-21T01:28:55.5959476Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:55.5960116Z env: 2024-08-21T01:28:55.5960434Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:55.5961961Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:55.5963536Z ##[endgroup] 2024-08-21T01:28:55.6003282Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:55.6068521Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-21T01:28:55.6069233Z with: 2024-08-21T01:28:55.6070852Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:55.6072605Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:55.6073223Z env: 2024-08-21T01:28:55.6073536Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:55.6073941Z ##[endgroup] 2024-08-21T01:28:55.6093879Z ##[group]Run set -x 2024-08-21T01:28:55.6094281Z set -x 2024-08-21T01:28:55.6094624Z set +e 2024-08-21T01:28:55.6094954Z  2024-08-21T01:28:55.6095254Z login() { 2024-08-21T01:28:55.6096142Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-21T01:28:55.6097042Z } 2024-08-21T01:28:55.6097352Z  2024-08-21T01:28:55.6097705Z retry () { 2024-08-21T01:28:55.6098160Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-21T01:28:55.6098680Z } 2024-08-21T01:28:55.6098988Z  2024-08-21T01:28:55.6099339Z retry login "${DOCKER_REGISTRY}" 2024-08-21T01:28:55.6099818Z  2024-08-21T01:28:55.6100124Z set -e 2024-08-21T01:28:55.6100687Z # ignore output since only exit code is used for conditional 2024-08-21T01:28:55.6101559Z # only pull docker image if it's not available locally 2024-08-21T01:28:55.6102505Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-21T01:28:55.6103361Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-21T01:28:55.6103884Z fi 2024-08-21T01:28:55.6116827Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:28:55.6117450Z env: 2024-08-21T01:28:55.6117749Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:28:55.6119261Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:55.6120999Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:55.6121639Z ##[endgroup] 2024-08-21T01:28:55.6155880Z + set +e 2024-08-21T01:28:55.6156600Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:55.6157392Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:55.6160523Z + aws ecr get-login-password --region us-east-1 2024-08-21T01:28:55.6161859Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-21T01:28:56.7677049Z WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json. 2024-08-21T01:28:56.7678170Z Configure a credential helper to remove this warning. See 2024-08-21T01:28:56.7679882Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-21T01:28:56.7680590Z 2024-08-21T01:28:56.7680736Z Login Succeeded 2024-08-21T01:28:56.7697539Z + set -e 2024-08-21T01:28:56.7699462Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:56.7915230Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:56.7918150Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:28:57.4038160Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-08-21T01:28:57.4064136Z 7a2c55901189: Pulling fs layer 2024-08-21T01:28:57.4065010Z 224fe954d725: Pulling fs layer 2024-08-21T01:28:57.4066217Z 75722010b82e: Pulling fs layer 2024-08-21T01:28:57.4067010Z d527cbbb87e3: Pulling fs layer 2024-08-21T01:28:57.4067985Z b57676e46aee: Pulling fs layer 2024-08-21T01:28:57.4069222Z a8c1e85b5e14: Pulling fs layer 2024-08-21T01:28:57.4069845Z a41a8d1c11c8: Pulling fs layer 2024-08-21T01:28:57.4070578Z 0c1227890755: Pulling fs layer 2024-08-21T01:28:57.4071305Z d8d1234baab3: Pulling fs layer 2024-08-21T01:28:57.4072031Z 7ed32bc8e469: Pulling fs layer 2024-08-21T01:28:57.4072822Z ec1e7978c1fe: Pulling fs layer 2024-08-21T01:28:57.4073637Z ac23c5fcc1ef: Pulling fs layer 2024-08-21T01:28:57.4074395Z ca426ffc9d2b: Pulling fs layer 2024-08-21T01:28:57.4075166Z 4373aa743caf: Pulling fs layer 2024-08-21T01:28:57.4075944Z 75b2abb16c68: Pulling fs layer 2024-08-21T01:28:57.4076404Z ef1e3ef1e85f: Pulling fs layer 2024-08-21T01:28:57.4076839Z 0bf61114d37b: Pulling fs layer 2024-08-21T01:28:57.4077285Z 2df849f72ba4: Pulling fs layer 2024-08-21T01:28:57.4077729Z 34c47d63c5de: Pulling fs layer 2024-08-21T01:28:57.4078194Z 1fef4bd19792: Pulling fs layer 2024-08-21T01:28:57.4078705Z 6efb96fc87ea: Pulling fs layer 2024-08-21T01:28:57.4079151Z b119c4fc6c58: Pulling fs layer 2024-08-21T01:28:57.4079673Z 33b639226eee: Pulling fs layer 2024-08-21T01:28:57.4080116Z d3547bd43146: Pulling fs layer 2024-08-21T01:28:57.4080555Z 5645520256fa: Pulling fs layer 2024-08-21T01:28:57.4080957Z 0c1227890755: Waiting 2024-08-21T01:28:57.4081342Z 829ed7d91e38: Pulling fs layer 2024-08-21T01:28:57.4081785Z 385d0421d67c: Pulling fs layer 2024-08-21T01:28:57.4082216Z 7cbfce999d76: Pulling fs layer 2024-08-21T01:28:57.4082640Z d527cbbb87e3: Waiting 2024-08-21T01:28:57.4083026Z 6658e558ebe4: Pulling fs layer 2024-08-21T01:28:57.4083463Z 33444a5a7e01: Pulling fs layer 2024-08-21T01:28:57.4083872Z a41a8d1c11c8: Waiting 2024-08-21T01:28:57.4084239Z b57676e46aee: Waiting 2024-08-21T01:28:57.4084618Z 7c47122f4e09: Pulling fs layer 2024-08-21T01:28:57.4085041Z a8c1e85b5e14: Waiting 2024-08-21T01:28:57.4085410Z 1811ff4d3556: Pulling fs layer 2024-08-21T01:28:57.4085878Z 7754ee068b33: Pulling fs layer 2024-08-21T01:28:57.4086298Z b119c4fc6c58: Waiting 2024-08-21T01:28:57.4086650Z 7ed32bc8e469: Waiting 2024-08-21T01:28:57.4087036Z 0479fbac3361: Pulling fs layer 2024-08-21T01:28:57.4087461Z 75b2abb16c68: Waiting 2024-08-21T01:28:57.4087827Z ec1e7978c1fe: Waiting 2024-08-21T01:28:57.4088195Z f1d6874dee9c: Pulling fs layer 2024-08-21T01:28:57.4088683Z afc429e87b59: Pulling fs layer 2024-08-21T01:28:57.4089120Z 019620471c25: Pulling fs layer 2024-08-21T01:28:57.4089534Z 4373aa743caf: Waiting 2024-08-21T01:28:57.4089883Z 33b639226eee: Waiting 2024-08-21T01:28:57.4090248Z 1fef4bd19792: Waiting 2024-08-21T01:28:57.4090636Z 27aeb1b2cd5e: Pulling fs layer 2024-08-21T01:28:57.4091048Z 6efb96fc87ea: Waiting 2024-08-21T01:28:57.4091430Z 829c0e19f4b7: Pulling fs layer 2024-08-21T01:28:57.4091851Z d3547bd43146: Waiting 2024-08-21T01:28:57.4092226Z 560cd6122a38: Pulling fs layer 2024-08-21T01:28:57.4092649Z 5a4eb19d7219: Pulling fs layer 2024-08-21T01:28:57.4093096Z da6a838a66c7: Pulling fs layer 2024-08-21T01:28:57.4093533Z cbb79a705353: Pulling fs layer 2024-08-21T01:28:57.4093940Z ac23c5fcc1ef: Waiting 2024-08-21T01:28:57.4094310Z 2df849f72ba4: Waiting 2024-08-21T01:28:57.4094697Z 4d4fa6802ab3: Pulling fs layer 2024-08-21T01:28:57.4095117Z 33444a5a7e01: Waiting 2024-08-21T01:28:57.4095480Z 0f085d4fa279: Pulling fs layer 2024-08-21T01:28:57.4095946Z 4a0412373305: Pulling fs layer 2024-08-21T01:28:57.4096359Z 1811ff4d3556: Waiting 2024-08-21T01:28:57.4096724Z 5a4eb19d7219: Waiting 2024-08-21T01:28:57.4097065Z 829c0e19f4b7: Waiting 2024-08-21T01:28:57.4097435Z 302c8c177944: Pulling fs layer 2024-08-21T01:28:57.4097866Z 02970347dff6: Pulling fs layer 2024-08-21T01:28:57.4098308Z ca426ffc9d2b: Waiting 2024-08-21T01:28:57.4098675Z 7c47122f4e09: Waiting 2024-08-21T01:28:57.4099050Z 916c5e858016: Pulling fs layer 2024-08-21T01:28:57.4099473Z 829ed7d91e38: Waiting 2024-08-21T01:28:57.4099816Z 385d0421d67c: Waiting 2024-08-21T01:28:57.4100168Z 0479fbac3361: Waiting 2024-08-21T01:28:57.4100822Z c42b3602a1fb: Pulling fs layer 2024-08-21T01:28:57.4101257Z afc429e87b59: Waiting 2024-08-21T01:28:57.4101626Z 11fad972ef6c: Pulling fs layer 2024-08-21T01:28:57.4102193Z cbb79a705353: Waiting 2024-08-21T01:28:57.4102579Z f957c1624481: Pulling fs layer 2024-08-21T01:28:57.4102989Z f1d6874dee9c: Waiting 2024-08-21T01:28:57.4103358Z 4a0412373305: Waiting 2024-08-21T01:28:57.4103730Z 768c31dca016: Pulling fs layer 2024-08-21T01:28:57.4104152Z 6658e558ebe4: Waiting 2024-08-21T01:28:57.4104501Z 302c8c177944: Waiting 2024-08-21T01:28:57.4104853Z f957c1624481: Waiting 2024-08-21T01:28:57.4105231Z 8c32d0ecebbe: Pulling fs layer 2024-08-21T01:28:57.4105641Z 768c31dca016: Waiting 2024-08-21T01:28:57.4106033Z c42b3602a1fb: Waiting 2024-08-21T01:28:57.4106421Z 4f4fb700ef54: Pulling fs layer 2024-08-21T01:28:57.4106848Z 11fad972ef6c: Waiting 2024-08-21T01:28:57.4107221Z 78ee77eddef2: Pulling fs layer 2024-08-21T01:28:57.4107952Z 37513e9b43d7: Pulling fs layer 2024-08-21T01:28:57.4108419Z 7cbfce999d76: Waiting 2024-08-21T01:28:57.4108817Z a7a5392e2bf6: Pulling fs layer 2024-08-21T01:28:57.4109246Z b10de02255c0: Pulling fs layer 2024-08-21T01:28:57.4109691Z e56eeff2db93: Pulling fs layer 2024-08-21T01:28:57.4110143Z 70ac3e38d0fd: Pulling fs layer 2024-08-21T01:28:57.4110549Z 7754ee068b33: Waiting 2024-08-21T01:28:57.4110938Z eb09cbf3e416: Pulling fs layer 2024-08-21T01:28:57.4111358Z 916c5e858016: Waiting 2024-08-21T01:28:57.4111732Z ee1273153675: Pulling fs layer 2024-08-21T01:28:57.4112156Z e22979f9e0ee: Pulling fs layer 2024-08-21T01:28:57.4112575Z b10de02255c0: Waiting 2024-08-21T01:28:57.4112933Z 0bf61114d37b: Waiting 2024-08-21T01:28:57.4113298Z eb09cbf3e416: Waiting 2024-08-21T01:28:57.4113642Z ee1273153675: Waiting 2024-08-21T01:28:57.4113993Z 5645520256fa: Waiting 2024-08-21T01:28:57.4114350Z 70ac3e38d0fd: Waiting 2024-08-21T01:28:57.4114698Z e22979f9e0ee: Waiting 2024-08-21T01:28:57.4115075Z 6c1f0aabce7c: Pulling fs layer 2024-08-21T01:28:57.4115524Z 7ebe49c3a23e: Pulling fs layer 2024-08-21T01:28:57.4115976Z 5d115b29b5d5: Pulling fs layer 2024-08-21T01:28:57.4116400Z 7f2d7a04fbe6: Pulling fs layer 2024-08-21T01:28:57.4116821Z 37513e9b43d7: Waiting 2024-08-21T01:28:57.4117206Z d79fd7d52698: Pulling fs layer 2024-08-21T01:28:57.4117616Z 7ebe49c3a23e: Waiting 2024-08-21T01:28:57.4118006Z 9752d6fbbcab: Pulling fs layer 2024-08-21T01:28:57.4118476Z 5d115b29b5d5: Waiting 2024-08-21T01:28:57.4118847Z 6c1f0aabce7c: Waiting 2024-08-21T01:28:57.4119220Z 83808b2eff4d: Pulling fs layer 2024-08-21T01:28:57.4119649Z 9752d6fbbcab: Waiting 2024-08-21T01:28:57.4120027Z a7a5392e2bf6: Waiting 2024-08-21T01:28:57.4120427Z 560cd6122a38: Waiting 2024-08-21T01:28:57.4120778Z 27aeb1b2cd5e: Waiting 2024-08-21T01:28:57.4121141Z 4f4fb700ef54: Waiting 2024-08-21T01:28:57.9666660Z 224fe954d725: Verifying Checksum 2024-08-21T01:28:57.9667759Z 224fe954d725: Download complete 2024-08-21T01:28:58.2556837Z 7a2c55901189: Download complete 2024-08-21T01:28:58.2637619Z d527cbbb87e3: Verifying Checksum 2024-08-21T01:28:58.2638256Z d527cbbb87e3: Download complete 2024-08-21T01:28:58.5452328Z b57676e46aee: Verifying Checksum 2024-08-21T01:28:58.5453319Z b57676e46aee: Download complete 2024-08-21T01:28:58.6248399Z 75722010b82e: Verifying Checksum 2024-08-21T01:28:58.6249378Z 75722010b82e: Download complete 2024-08-21T01:28:58.8721713Z a41a8d1c11c8: Verifying Checksum 2024-08-21T01:28:58.8722319Z a41a8d1c11c8: Download complete 2024-08-21T01:28:58.9012262Z 0c1227890755: Verifying Checksum 2024-08-21T01:28:58.9013182Z 0c1227890755: Download complete 2024-08-21T01:28:58.9493243Z 7a2c55901189: Pull complete 2024-08-21T01:28:59.1733841Z d8d1234baab3: Verifying Checksum 2024-08-21T01:28:59.1734656Z d8d1234baab3: Download complete 2024-08-21T01:28:59.1811264Z 224fe954d725: Pull complete 2024-08-21T01:28:59.5250779Z ec1e7978c1fe: Download complete 2024-08-21T01:28:59.8131795Z ac23c5fcc1ef: Download complete 2024-08-21T01:29:00.2039718Z 75722010b82e: Pull complete 2024-08-21T01:29:00.2683318Z d527cbbb87e3: Pull complete 2024-08-21T01:29:00.3272628Z b57676e46aee: Pull complete 2024-08-21T01:29:13.2673958Z a8c1e85b5e14: Verifying Checksum 2024-08-21T01:29:13.2674664Z a8c1e85b5e14: Download complete 2024-08-21T01:29:13.5696457Z 4373aa743caf: Download complete 2024-08-21T01:29:13.8588739Z 75b2abb16c68: Verifying Checksum 2024-08-21T01:29:13.8589713Z 75b2abb16c68: Download complete 2024-08-21T01:29:14.1943200Z ef1e3ef1e85f: Verifying Checksum 2024-08-21T01:29:14.1944109Z ef1e3ef1e85f: Download complete 2024-08-21T01:29:17.8945075Z 0bf61114d37b: Verifying Checksum 2024-08-21T01:29:17.8946032Z 0bf61114d37b: Download complete 2024-08-21T01:29:18.1605386Z 2df849f72ba4: Verifying Checksum 2024-08-21T01:29:18.1606384Z 2df849f72ba4: Download complete 2024-08-21T01:29:18.4728826Z 34c47d63c5de: Verifying Checksum 2024-08-21T01:29:18.4729667Z 34c47d63c5de: Download complete 2024-08-21T01:29:18.8100271Z 1fef4bd19792: Download complete 2024-08-21T01:29:19.3112731Z ca426ffc9d2b: Verifying Checksum 2024-08-21T01:29:19.3113554Z ca426ffc9d2b: Download complete 2024-08-21T01:29:19.6132851Z b119c4fc6c58: Download complete 2024-08-21T01:29:19.9086113Z 33b639226eee: Verifying Checksum 2024-08-21T01:29:19.9086879Z 33b639226eee: Download complete 2024-08-21T01:29:20.1988301Z d3547bd43146: Download complete 2024-08-21T01:29:20.4881753Z 5645520256fa: Verifying Checksum 2024-08-21T01:29:20.4882345Z 5645520256fa: Download complete 2024-08-21T01:29:20.7806604Z 829ed7d91e38: Verifying Checksum 2024-08-21T01:29:20.7807537Z 829ed7d91e38: Download complete 2024-08-21T01:29:24.6248422Z 385d0421d67c: Verifying Checksum 2024-08-21T01:29:24.6249391Z 385d0421d67c: Download complete 2024-08-21T01:29:24.9274532Z 7cbfce999d76: Download complete 2024-08-21T01:29:25.2328545Z 6658e558ebe4: Verifying Checksum 2024-08-21T01:29:25.2329499Z 6658e558ebe4: Download complete 2024-08-21T01:29:25.5402392Z 33444a5a7e01: Download complete 2024-08-21T01:29:25.8424686Z 7c47122f4e09: Download complete 2024-08-21T01:29:26.1419261Z 1811ff4d3556: Download complete 2024-08-21T01:29:26.7362466Z 7ed32bc8e469: Verifying Checksum 2024-08-21T01:29:26.7363424Z 7ed32bc8e469: Download complete 2024-08-21T01:29:27.0241465Z 0479fbac3361: Verifying Checksum 2024-08-21T01:29:27.0242373Z 0479fbac3361: Download complete 2024-08-21T01:29:27.3237431Z f1d6874dee9c: Download complete 2024-08-21T01:29:28.2617519Z a8c1e85b5e14: Pull complete 2024-08-21T01:29:28.3281690Z a41a8d1c11c8: Pull complete 2024-08-21T01:29:28.3571646Z afc429e87b59: Verifying Checksum 2024-08-21T01:29:28.3572253Z afc429e87b59: Download complete 2024-08-21T01:29:28.3937543Z 0c1227890755: Pull complete 2024-08-21T01:29:28.4494068Z d8d1234baab3: Pull complete 2024-08-21T01:29:28.6504300Z 019620471c25: Verifying Checksum 2024-08-21T01:29:28.6505265Z 019620471c25: Download complete 2024-08-21T01:29:28.9379515Z 27aeb1b2cd5e: Download complete 2024-08-21T01:29:29.8386799Z 829c0e19f4b7: Verifying Checksum 2024-08-21T01:29:29.8387965Z 829c0e19f4b7: Download complete 2024-08-21T01:29:30.1261891Z 560cd6122a38: Verifying Checksum 2024-08-21T01:29:30.1262680Z 560cd6122a38: Download complete 2024-08-21T01:29:30.4633023Z 5a4eb19d7219: Verifying Checksum 2024-08-21T01:29:30.4633849Z 5a4eb19d7219: Download complete 2024-08-21T01:29:30.7308339Z da6a838a66c7: Verifying Checksum 2024-08-21T01:29:43.8797961Z da6a838a66c7: Download complete 2024-08-21T01:29:43.8798724Z 7754ee068b33: Verifying Checksum 2024-08-21T01:29:43.8799192Z 7754ee068b33: Download complete 2024-08-21T01:29:44.3452236Z 4d4fa6802ab3: Verifying Checksum 2024-08-21T01:29:44.3453596Z 4d4fa6802ab3: Download complete 2024-08-21T01:29:47.6833373Z 0f085d4fa279: Verifying Checksum 2024-08-21T01:29:47.6834339Z 0f085d4fa279: Download complete 2024-08-21T01:29:49.4955543Z 4a0412373305: Verifying Checksum 2024-08-21T01:29:49.4956275Z 4a0412373305: Download complete 2024-08-21T01:29:49.7930830Z 302c8c177944: Verifying Checksum 2024-08-21T01:29:49.7931524Z 302c8c177944: Download complete 2024-08-21T01:29:50.0601907Z 02970347dff6: Download complete 2024-08-21T01:29:50.3438892Z 916c5e858016: Verifying Checksum 2024-08-21T01:29:50.3439978Z 916c5e858016: Download complete 2024-08-21T01:29:50.6288087Z c42b3602a1fb: Verifying Checksum 2024-08-21T01:29:50.6288810Z c42b3602a1fb: Download complete 2024-08-21T01:30:01.5469393Z 7ed32bc8e469: Pull complete 2024-08-21T01:30:02.0909491Z ec1e7978c1fe: Pull complete 2024-08-21T01:30:02.6113521Z ac23c5fcc1ef: Pull complete 2024-08-21T01:30:08.3952271Z ca426ffc9d2b: Pull complete 2024-08-21T01:30:08.4464636Z 4373aa743caf: Pull complete 2024-08-21T01:30:08.4974551Z 75b2abb16c68: Pull complete 2024-08-21T01:30:08.5486119Z ef1e3ef1e85f: Pull complete 2024-08-21T01:30:10.9542990Z 0bf61114d37b: Pull complete 2024-08-21T01:30:11.4282647Z 2df849f72ba4: Pull complete 2024-08-21T01:30:11.9459971Z 34c47d63c5de: Pull complete 2024-08-21T01:30:12.3837543Z 1fef4bd19792: Pull complete 2024-08-21T01:30:50.2867418Z 6efb96fc87ea: Verifying Checksum 2024-08-21T01:30:50.2868243Z 6efb96fc87ea: Download complete 2024-08-21T01:30:50.6792870Z f957c1624481: Verifying Checksum 2024-08-21T01:30:50.6793828Z f957c1624481: Download complete 2024-08-21T01:30:50.9883235Z 768c31dca016: Verifying Checksum 2024-08-21T01:30:50.9884168Z 768c31dca016: Download complete 2024-08-21T01:30:51.3214549Z 8c32d0ecebbe: Download complete 2024-08-21T01:30:51.4527038Z 4f4fb700ef54: Download complete 2024-08-21T01:30:51.7492333Z 78ee77eddef2: Verifying Checksum 2024-08-21T01:30:51.7493619Z 78ee77eddef2: Download complete 2024-08-21T01:30:52.0434208Z 37513e9b43d7: Verifying Checksum 2024-08-21T01:30:52.0435184Z 37513e9b43d7: Download complete 2024-08-21T01:30:52.6940695Z a7a5392e2bf6: Verifying Checksum 2024-08-21T01:30:52.6941596Z a7a5392e2bf6: Download complete 2024-08-21T01:30:52.9732523Z b10de02255c0: Download complete 2024-08-21T01:30:53.2661161Z e56eeff2db93: Verifying Checksum 2024-08-21T01:30:53.2662132Z e56eeff2db93: Download complete 2024-08-21T01:30:53.5559548Z 70ac3e38d0fd: Download complete 2024-08-21T01:30:53.8742316Z eb09cbf3e416: Verifying Checksum 2024-08-21T01:30:53.8742897Z eb09cbf3e416: Download complete 2024-08-21T01:30:54.5355320Z ee1273153675: Verifying Checksum 2024-08-21T01:30:54.5356241Z ee1273153675: Download complete 2024-08-21T01:30:54.8255492Z e22979f9e0ee: Download complete 2024-08-21T01:30:56.4501558Z 6c1f0aabce7c: Verifying Checksum 2024-08-21T01:30:56.4502413Z 6c1f0aabce7c: Download complete 2024-08-21T01:30:56.7883368Z 7ebe49c3a23e: Verifying Checksum 2024-08-21T01:30:56.7884119Z 7ebe49c3a23e: Download complete 2024-08-21T01:31:36.2739781Z 6efb96fc87ea: Pull complete 2024-08-21T01:31:36.3256683Z b119c4fc6c58: Pull complete 2024-08-21T01:31:36.3896768Z 33b639226eee: Pull complete 2024-08-21T01:31:36.4490598Z d3547bd43146: Pull complete 2024-08-21T01:31:36.5024076Z 5645520256fa: Pull complete 2024-08-21T01:31:36.5618744Z 829ed7d91e38: Pull complete 2024-08-21T01:31:40.3677702Z 5d115b29b5d5: Verifying Checksum 2024-08-21T01:31:40.3680387Z 5d115b29b5d5: Download complete 2024-08-21T01:31:40.4741991Z 385d0421d67c: Pull complete 2024-08-21T01:31:40.7536453Z 7cbfce999d76: Pull complete 2024-08-21T01:31:40.7567078Z 7f2d7a04fbe6: Verifying Checksum 2024-08-21T01:31:40.7567991Z 7f2d7a04fbe6: Download complete 2024-08-21T01:31:41.0541505Z d79fd7d52698: Verifying Checksum 2024-08-21T01:31:41.0542078Z d79fd7d52698: Download complete 2024-08-21T01:31:41.1037177Z 6658e558ebe4: Pull complete 2024-08-21T01:31:41.3605707Z 33444a5a7e01: Pull complete 2024-08-21T01:31:41.4751655Z 7c47122f4e09: Pull complete 2024-08-21T01:31:41.8525926Z 1811ff4d3556: Pull complete 2024-08-21T01:31:43.1005678Z 9752d6fbbcab: Verifying Checksum 2024-08-21T01:31:43.1006680Z 9752d6fbbcab: Download complete 2024-08-21T01:31:43.3965003Z 83808b2eff4d: Download complete 2024-08-21T01:31:49.9649650Z 7754ee068b33: Pull complete 2024-08-21T01:31:50.1021154Z 0479fbac3361: Pull complete 2024-08-21T01:31:50.1576544Z f1d6874dee9c: Pull complete 2024-08-21T01:31:50.9984749Z afc429e87b59: Pull complete 2024-08-21T01:31:51.0496723Z 019620471c25: Pull complete 2024-08-21T01:31:51.1006679Z 27aeb1b2cd5e: Pull complete 2024-08-21T01:31:51.4156747Z 829c0e19f4b7: Pull complete 2024-08-21T01:31:51.4676850Z 560cd6122a38: Pull complete 2024-08-21T01:31:51.4967554Z cbb79a705353: Verifying Checksum 2024-08-21T01:31:51.4968520Z cbb79a705353: Download complete 2024-08-21T01:31:51.5796826Z 5a4eb19d7219: Pull complete 2024-08-21T01:31:51.6316586Z da6a838a66c7: Pull complete 2024-08-21T01:32:02.1589717Z 11fad972ef6c: Verifying Checksum 2024-08-21T01:32:02.1590375Z 11fad972ef6c: Download complete 2024-08-21T01:32:33.0077299Z cbb79a705353: Pull complete 2024-08-21T01:32:33.0593491Z 4d4fa6802ab3: Pull complete 2024-08-21T01:32:33.1121093Z 0f085d4fa279: Pull complete 2024-08-21T01:32:34.7196243Z 4a0412373305: Pull complete 2024-08-21T01:32:35.2482822Z 302c8c177944: Pull complete 2024-08-21T01:32:35.4272076Z 02970347dff6: Pull complete 2024-08-21T01:32:35.5273376Z 916c5e858016: Pull complete 2024-08-21T01:32:35.5784715Z c42b3602a1fb: Pull complete 2024-08-21T01:34:01.7610360Z 11fad972ef6c: Pull complete 2024-08-21T01:34:01.8154634Z f957c1624481: Pull complete 2024-08-21T01:34:01.8679204Z 768c31dca016: Pull complete 2024-08-21T01:34:01.9704927Z 8c32d0ecebbe: Pull complete 2024-08-21T01:34:02.0234100Z 4f4fb700ef54: Pull complete 2024-08-21T01:34:02.0754395Z 78ee77eddef2: Pull complete 2024-08-21T01:34:02.1280378Z 37513e9b43d7: Pull complete 2024-08-21T01:34:02.3394492Z a7a5392e2bf6: Pull complete 2024-08-21T01:34:02.3908424Z b10de02255c0: Pull complete 2024-08-21T01:34:02.4408904Z e56eeff2db93: Pull complete 2024-08-21T01:34:02.4904362Z 70ac3e38d0fd: Pull complete 2024-08-21T01:34:02.5388581Z eb09cbf3e416: Pull complete 2024-08-21T01:34:02.9855142Z ee1273153675: Pull complete 2024-08-21T01:34:03.0354030Z e22979f9e0ee: Pull complete 2024-08-21T01:34:04.8309032Z 6c1f0aabce7c: Pull complete 2024-08-21T01:34:04.8806874Z 7ebe49c3a23e: Pull complete 2024-08-21T01:34:20.3458233Z 5d115b29b5d5: Pull complete 2024-08-21T01:34:20.3972154Z 7f2d7a04fbe6: Pull complete 2024-08-21T01:34:20.4564990Z d79fd7d52698: Pull complete 2024-08-21T01:34:21.2367878Z 9752d6fbbcab: Pull complete 2024-08-21T01:34:21.2849409Z 83808b2eff4d: Pull complete 2024-08-21T01:34:21.4944530Z Digest: sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-21T01:34:21.4974534Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:34:21.5003331Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:34:21.5092864Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-21T01:34:21.5094051Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-21T01:34:21.5108843Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:34:21.5109464Z env: 2024-08-21T01:34:21.5109769Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:34:21.5110182Z ##[endgroup] 2024-08-21T01:34:21.5345590Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-21T01:34:21.5346288Z with: 2024-08-21T01:34:21.5346623Z driver-version: 550.54.15 2024-08-21T01:34:21.5347045Z env: 2024-08-21T01:34:21.5347363Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:34:21.5348525Z ##[endgroup] 2024-08-21T01:34:21.5406003Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-21T01:34:21.5406706Z with: 2024-08-21T01:34:21.5407024Z timeout_minutes: 10 2024-08-21T01:34:21.5407395Z max_attempts: 3 2024-08-21T01:34:21.5453812Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-21T01:34:21.5500731Z retry_wait_seconds: 10 2024-08-21T01:34:21.5501169Z polling_interval_seconds: 1 2024-08-21T01:34:21.5501603Z warning_on_retry: true 2024-08-21T01:34:21.5502015Z continue_on_error: false 2024-08-21T01:34:21.5502420Z env: 2024-08-21T01:34:21.5502733Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:34:21.5503134Z DRIVER_VERSION: 550.54.15 2024-08-21T01:34:21.5503546Z ##[endgroup] 2024-08-21T01:34:21.6206748Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-21T01:34:21.6207578Z Before installing NVIDIA driver 2024-08-21T01:34:21.6208744Z + install_nvidia_driver_common 2024-08-21T01:34:21.6209576Z + echo 'Before installing NVIDIA driver' 2024-08-21T01:34:21.6210088Z + lspci 2024-08-21T01:34:21.6332456Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-21T01:34:21.6333471Z 00:01.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 03) 2024-08-21T01:34:21.6334414Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 2024-08-21T01:34:21.6335371Z 00:03.0 Non-VGA unclassified device: Red Hat, Inc. Virtio SCSI 2024-08-21T01:34:21.6336244Z 00:04.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) 2024-08-21T01:34:21.6337138Z 00:05.0 Ethernet controller: Red Hat, Inc. Virtio network device 2024-08-21T01:34:21.6337944Z 00:06.0 Unclassified device [00ff]: Red Hat, Inc. Virtio RNG 2024-08-21T01:34:21.6338746Z + lsmod 2024-08-21T01:34:21.6358919Z Module Size Used by 2024-08-21T01:34:21.6359446Z btrfs 1540096 0 2024-08-21T01:34:21.6359889Z blake2b_generic 20480 0 2024-08-21T01:34:21.6360350Z xor 24576 1 btrfs 2024-08-21T01:34:21.6360839Z zstd_compress 225280 1 btrfs 2024-08-21T01:34:21.6361430Z raid6_pq 122880 1 btrfs 2024-08-21T01:34:21.6361896Z ufs 106496 0 2024-08-21T01:34:21.6362335Z msdos 20480 0 2024-08-21T01:34:21.6362767Z xfs 1757184 0 2024-08-21T01:34:21.6363873Z ib_core 393216 0 2024-08-21T01:34:21.6364489Z nvidia_modeset 1343488 0 2024-08-21T01:34:21.6365199Z veth 32768 0 2024-08-21T01:34:21.6365869Z nvidia_uvm 4669440 0 2024-08-21T01:34:21.6366891Z nvidia 54018048 7 nvidia_uvm,nvidia_modeset 2024-08-21T01:34:21.6367568Z drm 618496 1 nvidia 2024-08-21T01:34:21.6368169Z xt_conntrack 16384 1 2024-08-21T01:34:21.6368623Z xt_MASQUERADE 20480 1 2024-08-21T01:34:21.6369055Z xfrm_user 40960 1 2024-08-21T01:34:21.6369527Z xfrm_algo 16384 1 xfrm_user 2024-08-21T01:34:21.6370147Z xt_addrtype 16384 2 2024-08-21T01:34:21.6370914Z iptable_filter 16384 1 2024-08-21T01:34:21.6371490Z iptable_nat 16384 1 2024-08-21T01:34:21.6372141Z nf_nat 49152 2 iptable_nat,xt_MASQUERADE 2024-08-21T01:34:21.6373030Z bpfilter 16384 0 2024-08-21T01:34:21.6373909Z br_netfilter 32768 0 2024-08-21T01:34:21.6374844Z bridge 307200 1 br_netfilter 2024-08-21T01:34:21.6375386Z stp 16384 1 bridge 2024-08-21T01:34:21.6376087Z llc 16384 2 bridge,stp 2024-08-21T01:34:21.6376986Z aufs 270336 0 2024-08-21T01:34:21.6377429Z overlay 151552 0 2024-08-21T01:34:21.6377943Z nls_iso8859_1 16384 1 2024-08-21T01:34:21.6378858Z dm_multipath 40960 0 2024-08-21T01:34:21.6379406Z scsi_dh_rdac 16384 0 2024-08-21T01:34:21.6379982Z scsi_dh_emc 16384 0 2024-08-21T01:34:21.6380744Z scsi_dh_alua 20480 0 2024-08-21T01:34:21.6381526Z binfmt_misc 24576 1 2024-08-21T01:34:21.6382371Z crct10dif_pclmul 16384 1 2024-08-21T01:34:21.6383021Z crc32_pclmul 16384 0 2024-08-21T01:34:21.6383450Z ghash_clmulni_intel 16384 0 2024-08-21T01:34:21.6383898Z sha256_ssse3 32768 0 2024-08-21T01:34:21.6384328Z sha1_ssse3 32768 0 2024-08-21T01:34:21.6384773Z aesni_intel 376832 0 2024-08-21T01:34:21.6386295Z input_leds 16384 0 2024-08-21T01:34:21.6386836Z crypto_simd 16384 1 aesni_intel 2024-08-21T01:34:21.6387364Z psmouse 176128 0 2024-08-21T01:34:21.6388334Z virtio_net 61440 0 2024-08-21T01:34:21.6388970Z cryptd 24576 2 crypto_simd,ghash_clmulni_intel 2024-08-21T01:34:21.6389603Z net_failover 20480 1 virtio_net 2024-08-21T01:34:21.6390210Z failover 16384 1 net_failover 2024-08-21T01:34:21.6390730Z serio_raw 20480 0 2024-08-21T01:34:21.6391155Z sch_fq_codel 24576 13 2024-08-21T01:34:21.6391604Z efi_pstore 16384 0 2024-08-21T01:34:21.6392043Z virtio_rng 16384 0 2024-08-21T01:34:21.6392595Z ip_tables 32768 2 iptable_filter,iptable_nat 2024-08-21T01:34:21.6393564Z x_tables 53248 6 xt_conntrack,iptable_filter,xt_addrtype,ip_tables,iptable_nat,xt_MASQUERADE 2024-08-21T01:34:21.6394448Z autofs4 49152 2 2024-08-21T01:34:21.6394874Z + modinfo nvidia 2024-08-21T01:34:21.6395663Z filename: /lib/modules/5.15.0-1062-gcp/kernel/drivers/video/nvidia.ko 2024-08-21T01:34:21.6396478Z alias: char-major-195-* 2024-08-21T01:34:21.6396940Z version: 550.54.15 2024-08-21T01:34:21.6397340Z supported: external 2024-08-21T01:34:21.6397734Z license: NVIDIA 2024-08-21T01:34:21.6398165Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-21T01:34:21.6398755Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-21T01:34:21.6399308Z srcversion: 833721318DA517F0C2FEC97 2024-08-21T01:34:21.6399868Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-21T01:34:21.6400461Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-21T01:34:21.6401104Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-21T01:34:21.6401636Z depends: drm 2024-08-21T01:34:21.6401987Z retpoline: Y 2024-08-21T01:34:21.6402351Z name: nvidia 2024-08-21T01:34:21.6403236Z vermagic: 5.15.0-1062-gcp SMP mod_unload modversions 2024-08-21T01:34:21.6404037Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-21T01:34:21.6404832Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-21T01:34:21.6405585Z parm: NVreg_ResmanDebugLevel:int 2024-08-21T01:34:21.6406158Z parm: NVreg_RmLogonRC:int 2024-08-21T01:34:21.6406676Z parm: NVreg_ModifyDeviceFiles:int 2024-08-21T01:34:21.6407217Z parm: NVreg_DeviceFileUID:int 2024-08-21T01:34:21.6407739Z parm: NVreg_DeviceFileGID:int 2024-08-21T01:34:21.6408265Z parm: NVreg_DeviceFileMode:int 2024-08-21T01:34:21.6408916Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-21T01:34:21.6409599Z parm: NVreg_UsePageAttributeTable:int 2024-08-21T01:34:21.6410177Z parm: NVreg_EnablePCIeGen3:int 2024-08-21T01:34:21.6410693Z parm: NVreg_EnableMSI:int 2024-08-21T01:34:21.6411188Z parm: NVreg_TCEBypassMode:int 2024-08-21T01:34:21.6411726Z parm: NVreg_EnableStreamMemOPs:int 2024-08-21T01:34:21.6412372Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-21T01:34:21.6413102Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-21T01:34:21.6413925Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-21T01:34:21.6414686Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-21T01:34:21.6415437Z parm: NVreg_DynamicPowerManagement:int 2024-08-21T01:34:21.6416248Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-21T01:34:21.6416980Z parm: NVreg_EnableGpuFirmware:int 2024-08-21T01:34:21.6417584Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-21T01:34:21.6418241Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-21T01:34:21.6418914Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-21T01:34:21.6419508Z parm: NVreg_MemoryPoolSize:int 2024-08-21T01:34:21.6420071Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-21T01:34:21.6420654Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-21T01:34:21.6421225Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-21T01:34:21.6421751Z parm: NVreg_NvLinkDisable:int 2024-08-21T01:34:21.6422370Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-21T01:34:21.6423017Z parm: NVreg_RegisterPCIDriver:int 2024-08-21T01:34:21.6423593Z parm: NVreg_EnableResizableBar:int 2024-08-21T01:34:21.6424171Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-21T01:34:21.6424785Z parm: NVreg_EnableNonblockingOpen:int 2024-08-21T01:34:21.6425375Z parm: NVreg_RegistryDwords:charp 2024-08-21T01:34:21.6426066Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-21T01:34:21.6426638Z parm: NVreg_RmMsg:charp 2024-08-21T01:34:21.6427129Z parm: NVreg_GpuBlacklist:charp 2024-08-21T01:34:21.6427994Z parm: NVreg_TemporaryFilePath:charp 2024-08-21T01:34:21.6428570Z parm: NVreg_ExcludedGpus:charp 2024-08-21T01:34:21.6429127Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-21T01:34:21.6429709Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-21T01:34:21.6430299Z parm: NVreg_ImexChannelCount:int 2024-08-21T01:34:21.6430833Z parm: rm_firmware_active:charp 2024-08-21T01:34:21.6431329Z + HAS_NVIDIA_DRIVER=0 2024-08-21T01:34:21.6431821Z ++ command -v nvidia-smi 2024-08-21T01:34:21.6432301Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-21T01:34:21.6432724Z + set +e 2024-08-21T01:34:21.6433400Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-21T01:34:21.6707191Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-21T01:34:21.6708150Z + NVIDIA_SMI_STATUS=0 2024-08-21T01:34:21.6709522Z + '[' 0 -ne 0 ']' 2024-08-21T01:34:21.6710230Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-21T01:34:21.6710931Z + HAS_NVIDIA_DRIVER=1 2024-08-21T01:34:21.6712651Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-21T01:34:21.6713585Z + set -e 2024-08-21T01:34:21.6713924Z + '[' 1 -eq 0 ']' 2024-08-21T01:34:21.6714659Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-21T01:34:21.6715536Z + post_install_nvidia_driver_common 2024-08-21T01:34:21.6716022Z + sudo modprobe nvidia 2024-08-21T01:34:21.6837022Z + echo 'After installing NVIDIA driver' 2024-08-21T01:34:21.6837580Z + lspci 2024-08-21T01:34:21.6839356Z After installing NVIDIA driver 2024-08-21T01:34:21.6947689Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-21T01:34:21.6948905Z 00:01.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 03) 2024-08-21T01:34:21.6949808Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) 2024-08-21T01:34:21.6950770Z 00:03.0 Non-VGA unclassified device: Red Hat, Inc. Virtio SCSI 2024-08-21T01:34:21.6951569Z 00:04.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) 2024-08-21T01:34:21.6952432Z 00:05.0 Ethernet controller: Red Hat, Inc. Virtio network device 2024-08-21T01:34:21.6953243Z 00:06.0 Unclassified device [00ff]: Red Hat, Inc. Virtio RNG 2024-08-21T01:34:21.6953848Z + lsmod 2024-08-21T01:34:21.6971994Z Module Size Used by 2024-08-21T01:34:21.6972789Z btrfs 1540096 0 2024-08-21T01:34:21.6973702Z blake2b_generic 20480 0 2024-08-21T01:34:21.6974530Z xor 24576 1 btrfs 2024-08-21T01:34:21.6975154Z zstd_compress 225280 1 btrfs 2024-08-21T01:34:21.6975623Z raid6_pq 122880 1 btrfs 2024-08-21T01:34:21.6976140Z ufs 106496 0 2024-08-21T01:34:21.6976574Z msdos 20480 0 2024-08-21T01:34:21.6976992Z xfs 1757184 0 2024-08-21T01:34:21.6977421Z ib_core 393216 0 2024-08-21T01:34:21.6977867Z nvidia_modeset 1343488 0 2024-08-21T01:34:21.6978301Z veth 32768 0 2024-08-21T01:34:21.6978747Z nvidia_uvm 4669440 0 2024-08-21T01:34:21.6979299Z nvidia 54018048 7 nvidia_uvm,nvidia_modeset 2024-08-21T01:34:21.6979885Z drm 618496 1 nvidia 2024-08-21T01:34:21.6980364Z xt_conntrack 16384 1 2024-08-21T01:34:21.6980806Z xt_MASQUERADE 20480 1 2024-08-21T01:34:21.6981242Z xfrm_user 40960 1 2024-08-21T01:34:21.6981703Z xfrm_algo 16384 1 xfrm_user 2024-08-21T01:34:21.6982191Z xt_addrtype 16384 2 2024-08-21T01:34:21.6982634Z iptable_filter 16384 1 2024-08-21T01:34:21.6983080Z iptable_nat 16384 1 2024-08-21T01:34:21.6983611Z nf_nat 49152 2 iptable_nat,xt_MASQUERADE 2024-08-21T01:34:21.6984171Z bpfilter 16384 0 2024-08-21T01:34:21.6984609Z br_netfilter 32768 0 2024-08-21T01:34:21.6985086Z bridge 307200 1 br_netfilter 2024-08-21T01:34:21.6985607Z stp 16384 1 bridge 2024-08-21T01:34:21.6986133Z llc 16384 2 bridge,stp 2024-08-21T01:34:21.6986631Z aufs 270336 0 2024-08-21T01:34:21.6987063Z overlay 151552 0 2024-08-21T01:34:21.6987709Z nls_iso8859_1 16384 1 2024-08-21T01:34:21.6988244Z dm_multipath 40960 0 2024-08-21T01:34:21.6988699Z scsi_dh_rdac 16384 0 2024-08-21T01:34:21.6989139Z scsi_dh_emc 16384 0 2024-08-21T01:34:21.6989660Z scsi_dh_alua 20480 0 2024-08-21T01:34:21.6990322Z binfmt_misc 24576 1 2024-08-21T01:34:21.6990762Z crct10dif_pclmul 16384 1 2024-08-21T01:34:21.6991202Z crc32_pclmul 16384 0 2024-08-21T01:34:21.6991959Z ghash_clmulni_intel 16384 0 2024-08-21T01:34:21.6992705Z sha256_ssse3 32768 0 2024-08-21T01:34:21.6993557Z sha1_ssse3 32768 0 2024-08-21T01:34:21.6994273Z aesni_intel 376832 0 2024-08-21T01:34:21.6994697Z input_leds 16384 0 2024-08-21T01:34:21.6995162Z crypto_simd 16384 1 aesni_intel 2024-08-21T01:34:21.6995987Z psmouse 176128 0 2024-08-21T01:34:21.6996436Z virtio_net 61440 0 2024-08-21T01:34:21.6996987Z cryptd 24576 2 crypto_simd,ghash_clmulni_intel 2024-08-21T01:34:21.6997631Z net_failover 20480 1 virtio_net 2024-08-21T01:34:21.6998196Z failover 16384 1 net_failover 2024-08-21T01:34:21.6998716Z serio_raw 20480 0 2024-08-21T01:34:21.6999145Z sch_fq_codel 24576 13 2024-08-21T01:34:21.6999595Z efi_pstore 16384 0 2024-08-21T01:34:21.7000031Z virtio_rng 16384 0 2024-08-21T01:34:21.7000569Z ip_tables 32768 2 iptable_filter,iptable_nat 2024-08-21T01:34:21.7001549Z x_tables 53248 6 xt_conntrack,iptable_filter,xt_addrtype,ip_tables,iptable_nat,xt_MASQUERADE 2024-08-21T01:34:21.7002408Z autofs4 49152 2 2024-08-21T01:34:21.7002833Z + modinfo nvidia 2024-08-21T01:34:21.7003580Z filename: /lib/modules/5.15.0-1062-gcp/kernel/drivers/video/nvidia.ko 2024-08-21T01:34:21.7004369Z alias: char-major-195-* 2024-08-21T01:34:21.7004826Z version: 550.54.15 2024-08-21T01:34:21.7005228Z supported: external 2024-08-21T01:34:21.7005610Z license: NVIDIA 2024-08-21T01:34:21.7006243Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-21T01:34:21.7006872Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-21T01:34:21.7007432Z srcversion: 833721318DA517F0C2FEC97 2024-08-21T01:34:21.7007980Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-21T01:34:21.7008588Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-21T01:34:21.7009187Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-21T01:34:21.7009712Z depends: drm 2024-08-21T01:34:21.7010063Z retpoline: Y 2024-08-21T01:34:21.7010416Z name: nvidia 2024-08-21T01:34:21.7011015Z vermagic: 5.15.0-1062-gcp SMP mod_unload modversions 2024-08-21T01:34:21.7011753Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-21T01:34:21.7012547Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-21T01:34:21.7013291Z parm: NVreg_ResmanDebugLevel:int 2024-08-21T01:34:21.7013827Z parm: NVreg_RmLogonRC:int 2024-08-21T01:34:21.7014351Z parm: NVreg_ModifyDeviceFiles:int 2024-08-21T01:34:21.7014885Z parm: NVreg_DeviceFileUID:int 2024-08-21T01:34:21.7015408Z parm: NVreg_DeviceFileGID:int 2024-08-21T01:34:21.7015971Z parm: NVreg_DeviceFileMode:int 2024-08-21T01:34:21.7016605Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-21T01:34:21.7017299Z parm: NVreg_UsePageAttributeTable:int 2024-08-21T01:34:21.7017880Z parm: NVreg_EnablePCIeGen3:int 2024-08-21T01:34:21.7018394Z parm: NVreg_EnableMSI:int 2024-08-21T01:34:21.7018875Z parm: NVreg_TCEBypassMode:int 2024-08-21T01:34:21.7019425Z parm: NVreg_EnableStreamMemOPs:int 2024-08-21T01:34:21.7020074Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-21T01:34:21.7020800Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-21T01:34:21.7021480Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-21T01:34:21.7022238Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-21T01:34:21.7022990Z parm: NVreg_DynamicPowerManagement:int 2024-08-21T01:34:21.7023762Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-21T01:34:21.7024488Z parm: NVreg_EnableGpuFirmware:int 2024-08-21T01:34:21.7025085Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-21T01:34:21.7025739Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-21T01:34:21.7026450Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-21T01:34:21.7027046Z parm: NVreg_MemoryPoolSize:int 2024-08-21T01:34:21.7027940Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-21T01:34:21.7028555Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-21T01:34:21.7029129Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-21T01:34:21.7029908Z parm: NVreg_NvLinkDisable:int 2024-08-21T01:34:21.7030531Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-21T01:34:21.7031178Z parm: NVreg_RegisterPCIDriver:int 2024-08-21T01:34:21.7031767Z parm: NVreg_EnableResizableBar:int 2024-08-21T01:34:21.7032343Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-21T01:34:21.7032952Z parm: NVreg_EnableNonblockingOpen:int 2024-08-21T01:34:21.7033542Z parm: NVreg_RegistryDwords:charp 2024-08-21T01:34:21.7034135Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-21T01:34:21.7034784Z parm: NVreg_RmMsg:charp 2024-08-21T01:34:21.7035276Z parm: NVreg_GpuBlacklist:charp 2024-08-21T01:34:21.7035839Z parm: NVreg_TemporaryFilePath:charp 2024-08-21T01:34:21.7036448Z parm: NVreg_ExcludedGpus:charp 2024-08-21T01:34:21.7036993Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-21T01:34:21.7037575Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-21T01:34:21.7038153Z parm: NVreg_ImexChannelCount:int 2024-08-21T01:34:21.7038684Z parm: rm_firmware_active:charp 2024-08-21T01:34:21.7039161Z + set +e 2024-08-21T01:34:21.7039555Z + nvidia-smi 2024-08-21T01:34:21.7215766Z Wed Aug 21 01:34:21 2024 2024-08-21T01:34:21.7216724Z +-----------------------------------------------------------------------------------------+ 2024-08-21T01:34:21.7217786Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-21T01:34:21.7218806Z |-----------------------------------------+------------------------+----------------------+ 2024-08-21T01:34:21.7219811Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-21T01:34:21.7220959Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-21T01:34:21.7221823Z | | | MIG M. | 2024-08-21T01:34:21.7222514Z |=========================================+========================+======================| 2024-08-21T01:34:21.7521431Z | 0 NVIDIA A100-SXM4-40GB On | 00000000:00:04.0 Off | 0 | 2024-08-21T01:34:21.7522563Z | N/A 33C P0 45W / 400W | 0MiB / 40960MiB | 0% Default | 2024-08-21T01:34:21.7523537Z | | | Disabled | 2024-08-21T01:34:21.7524959Z +-----------------------------------------+------------------------+----------------------+ 2024-08-21T01:34:21.7525816Z 2024-08-21T01:34:21.7526746Z +-----------------------------------------------------------------------------------------+ 2024-08-21T01:34:21.7527553Z | Processes: | 2024-08-21T01:34:21.7528487Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-21T01:34:21.7529347Z | ID ID Usage | 2024-08-21T01:34:21.7530058Z |=========================================================================================| 2024-08-21T01:34:21.7530891Z | No running processes found | 2024-08-21T01:34:21.7531883Z +-----------------------------------------------------------------------------------------+ 2024-08-21T01:34:22.1522491Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-21T01:34:22.1717591Z NVIDIA A100-SXM4-40GB 2024-08-21T01:34:22.1793204Z + NVIDIA_SMI_STATUS=0 2024-08-21T01:34:22.1793949Z + '[' 0 -eq 0 ']' 2024-08-21T01:34:22.1794452Z + echo 'INFO: Ignoring allowed status 0' 2024-08-21T01:34:22.1794962Z + set -e 2024-08-21T01:34:22.1795297Z INFO: Ignoring allowed status 0 2024-08-21T01:34:22.1803366Z == Installing nvidia container toolkit for ubuntu20.04 == 2024-08-21T01:34:22.1811088Z ++ dpkg-query -W '--showformat=${db:Status-Status}' nvidia-docker2 2024-08-21T01:34:22.1918908Z + status=installed 2024-08-21T01:34:22.1919630Z + '[' '!' 0 = 0 ']' 2024-08-21T01:34:22.1920109Z + '[' '!' installed = installed ']' 2024-08-21T01:34:22.2007729Z nvidia-persistenced failed to initialize. Check syslog for more details. 2024-08-21T01:34:22.2221651Z Wed Aug 21 01:34:22 2024 2024-08-21T01:34:22.2223193Z +-----------------------------------------------------------------------------------------+ 2024-08-21T01:34:22.2224296Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-21T01:34:22.2225311Z |-----------------------------------------+------------------------+----------------------+ 2024-08-21T01:34:22.2226320Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-21T01:34:22.2227620Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-21T01:34:22.2228547Z | | | MIG M. | 2024-08-21T01:34:22.2229229Z |=========================================+========================+======================| 2024-08-21T01:34:22.2454759Z | 0 NVIDIA A100-SXM4-40GB On | 00000000:00:04.0 Off | 0 | 2024-08-21T01:34:22.2456260Z | N/A 33C P0 61W / 400W | 0MiB / 40960MiB | 0% Default | 2024-08-21T01:34:22.2457820Z | | | Disabled | 2024-08-21T01:34:22.2459134Z +-----------------------------------------+------------------------+----------------------+ 2024-08-21T01:34:22.2459858Z 2024-08-21T01:34:22.2460691Z +-----------------------------------------------------------------------------------------+ 2024-08-21T01:34:22.2461514Z | Processes: | 2024-08-21T01:34:22.2462409Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-21T01:34:22.2463293Z | ID ID Usage | 2024-08-21T01:34:22.2464020Z |=========================================================================================| 2024-08-21T01:34:22.2464834Z | No running processes found | 2024-08-21T01:34:22.2465855Z +-----------------------------------------------------------------------------------------+ 2024-08-21T01:34:22.6196532Z Command completed after 1 attempt(s). 2024-08-21T01:34:22.6284023Z ##[group]Run sudo nvidia-smi -pm 1 2024-08-21T01:34:22.6284580Z sudo nvidia-smi -pm 1 2024-08-21T01:34:22.6285075Z sudo nvidia-smi -ac 1215,1410 2024-08-21T01:34:22.6285565Z nvidia-smi 2024-08-21T01:34:22.6298879Z shell: /usr/bin/bash -e {0} 2024-08-21T01:34:22.6299310Z env: 2024-08-21T01:34:22.6299632Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:34:22.6300181Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:34:22.6300782Z ##[endgroup] 2024-08-21T01:34:22.6783204Z Persistence mode is already Enabled for GPU 00000000:00:04.0. 2024-08-21T01:34:22.6783908Z All done. 2024-08-21T01:34:22.7343088Z Applications clocks set to "(MEM 1215, SM 1410)" for GPU 00000000:00:04.0 2024-08-21T01:34:22.7343859Z All done. 2024-08-21T01:34:22.7583266Z Wed Aug 21 01:34:22 2024 2024-08-21T01:34:22.7584474Z +-----------------------------------------------------------------------------------------+ 2024-08-21T01:34:22.7585622Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-21T01:34:22.7586698Z |-----------------------------------------+------------------------+----------------------+ 2024-08-21T01:34:22.7587878Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-21T01:34:22.7589029Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-21T01:34:22.7589904Z | | | MIG M. | 2024-08-21T01:34:22.7590602Z |=========================================+========================+======================| 2024-08-21T01:34:22.7815239Z | 0 NVIDIA A100-SXM4-40GB On | 00000000:00:04.0 Off | 0 | 2024-08-21T01:34:22.7816419Z | N/A 34C P0 68W / 400W | 0MiB / 40960MiB | 0% Default | 2024-08-21T01:34:22.7817252Z | | | Disabled | 2024-08-21T01:34:22.7818127Z +-----------------------------------------+------------------------+----------------------+ 2024-08-21T01:34:22.7818863Z 2024-08-21T01:34:22.7819701Z +-----------------------------------------------------------------------------------------+ 2024-08-21T01:34:22.7820498Z | Processes: | 2024-08-21T01:34:22.7821812Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-21T01:34:22.7822684Z | ID ID Usage | 2024-08-21T01:34:22.7823374Z |=========================================================================================| 2024-08-21T01:34:22.7824202Z | No running processes found | 2024-08-21T01:34:22.7825200Z +-----------------------------------------------------------------------------------------+ 2024-08-21T01:34:23.0352800Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-21T01:34:23.0353800Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-21T01:34:23.0354669Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-21T01:34:23.0355472Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-21T01:34:23.0368860Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:34:23.0369487Z env: 2024-08-21T01:34:23.0369814Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:34:23.0370353Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:34:23.0370953Z ##[endgroup] 2024-08-21T01:34:24.2477837Z Requirement already satisfied: psutil==5.9.1 in /home/ubuntu/.local/lib/python3.8/site-packages (5.9.1) 2024-08-21T01:34:24.2567068Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ubuntu/.local/lib/python3.8/site-packages (11.525.84) 2024-08-21T01:34:24.4798231Z Prepare all required actions 2024-08-21T01:34:24.4798902Z Getting action download info 2024-08-21T01:34:24.6278082Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-21T01:34:25.3523934Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-21T01:34:25.8990134Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-21T01:34:25.8990745Z with: 2024-08-21T01:34:25.8991147Z name: linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-08-21T01:34:25.8991702Z s3-bucket: gha-artifacts 2024-08-21T01:34:25.8992102Z env: 2024-08-21T01:34:25.8992402Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:34:25.8992947Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:34:25.8993534Z ##[endgroup] 2024-08-21T01:34:25.9555738Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-21T01:34:25.9556379Z with: 2024-08-21T01:34:25.9556858Z name: linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-08-21T01:34:25.9557414Z s3-bucket: gha-artifacts 2024-08-21T01:34:25.9557836Z region: us-east-1 2024-08-21T01:34:25.9558186Z env: 2024-08-21T01:34:25.9558496Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:34:25.9559040Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:34:25.9559615Z ##[endgroup] 2024-08-21T01:34:26.5495218Z (node:1151407) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-21T01:34:26.5496241Z 2024-08-21T01:34:26.5496564Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-21T01:34:26.5497484Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-21T01:34:26.5499087Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-21T01:34:26.6941807Z Found 1 objects with prefix pytorch/pytorch/10479310721/linux-focal-cuda12.1-py3.10-gcc9-sm80/ 2024-08-21T01:34:26.6943327Z Starting download (1/1): /home/weiwangmeta/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-21T01:34:52.6307754Z Finished download (1/1): /home/weiwangmeta/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-21T01:34:52.6314853Z Artifact download has finished successfully 2024-08-21T01:34:52.6647907Z ##[group]Run unzip -o artifacts.zip 2024-08-21T01:34:52.6648467Z unzip -o artifacts.zip 2024-08-21T01:34:52.6662415Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:34:52.6663873Z env: 2024-08-21T01:34:52.6664228Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:34:52.6664821Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:34:52.6665446Z ##[endgroup] 2024-08-21T01:34:52.6725704Z Archive: artifacts.zip 2024-08-21T01:34:52.6727990Z creating: dist/ 2024-08-21T01:34:55.1122943Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-21T01:34:55.1123837Z creating: build/custom_test_artifacts/ 2024-08-21T01:34:55.1124614Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-21T01:34:55.1125698Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-21T01:34:55.1127437Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-21T01:34:55.1133245Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-21T01:34:55.1134558Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-21T01:34:55.1135815Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-21T01:34:55.1137355Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-21T01:34:55.1138731Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-21T01:34:55.1140201Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-21T01:34:55.1142115Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-21T01:34:55.1143663Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-21T01:34:55.1144982Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-21T01:34:55.1146606Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-21T01:34:55.1148407Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-21T01:34:55.1149926Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-21T01:34:55.1151493Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-21T01:34:55.1153038Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-21T01:34:55.1154597Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-21T01:34:55.1155994Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-21T01:34:55.1157303Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-21T01:34:55.1199874Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-21T01:34:55.1246242Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-21T01:34:55.1248279Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-21T01:34:55.1300976Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-21T01:34:55.1302911Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-21T01:34:55.1304850Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-21T01:34:55.1306955Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-21T01:34:55.1309005Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-21T01:34:55.1311324Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-21T01:34:55.1313311Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-21T01:34:55.1315170Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-21T01:34:55.1317047Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-21T01:34:55.1318803Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-21T01:34:55.1320511Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-21T01:34:55.1322125Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-21T01:34:55.1323775Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-21T01:34:55.1325368Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-21T01:34:55.1327023Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-21T01:34:55.1387805Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-21T01:34:55.1460034Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-21T01:34:55.1461631Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-21T01:34:55.1462951Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-21T01:34:55.1464093Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-21T01:34:55.1465281Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-21T01:34:55.1466569Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-21T01:34:55.1468315Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-21T01:34:55.1469860Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-21T01:34:55.1471328Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-21T01:34:55.1472699Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-21T01:34:55.1474103Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-21T01:34:55.1475536Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-21T01:34:55.1477009Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-21T01:34:55.1478428Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-21T01:34:55.1479812Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-21T01:34:55.1494546Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-21T01:34:55.1666440Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-21T01:34:55.1667980Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-21T01:34:55.1669735Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-21T01:34:55.1671332Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-21T01:34:55.1673175Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-21T01:34:55.1674642Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-21T01:34:55.1676200Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-21T01:34:55.1677726Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-21T01:34:55.1679222Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-21T01:34:55.1680725Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-21T01:34:55.1682209Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-21T01:34:55.1698122Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-21T01:34:55.1797986Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-21T01:34:55.1799593Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-21T01:34:55.1801324Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-21T01:34:55.1802891Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-21T01:34:55.1804101Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-21T01:34:55.1805336Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-21T01:34:55.1806894Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-21T01:34:55.1808018Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-21T01:34:55.1809123Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-21T01:34:55.1810209Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-21T01:34:55.1950514Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-21T01:34:55.2025470Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-21T01:34:55.2026771Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-21T01:34:55.2027876Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-21T01:34:55.2028948Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-21T01:34:55.2036134Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-21T01:34:55.2037384Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-21T01:34:55.2038599Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-21T01:34:55.2040135Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-21T01:34:55.2041501Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-21T01:34:55.2042949Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-21T01:34:55.2044394Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-21T01:34:55.2045892Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-21T01:34:55.2047482Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-21T01:34:55.2049008Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-21T01:34:55.2050815Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-21T01:34:55.2052658Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-21T01:34:55.2054413Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-21T01:34:55.2056042Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-21T01:34:55.2057567Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-21T01:34:55.2058973Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-21T01:34:55.2060278Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-21T01:34:55.2103749Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-21T01:34:55.2151536Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-21T01:34:55.2153508Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-21T01:34:55.2207294Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-21T01:34:55.2209292Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-21T01:34:55.2211564Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-21T01:34:55.2213548Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-21T01:34:55.2215472Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-21T01:34:55.2217644Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-21T01:34:55.2219571Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-21T01:34:55.2221416Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-21T01:34:55.2223230Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-21T01:34:55.2224918Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-21T01:34:55.2226638Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-21T01:34:55.2228613Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-21T01:34:55.2230258Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-21T01:34:55.2231828Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-21T01:34:55.2233422Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-21T01:34:55.2292019Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-21T01:34:55.2365059Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-21T01:34:55.2367008Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-21T01:34:55.2368324Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-21T01:34:55.2369428Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-21T01:34:55.2371084Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-21T01:34:55.2372372Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-21T01:34:55.2373794Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-21T01:34:55.2375366Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-21T01:34:55.2376932Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-21T01:34:55.2378336Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-21T01:34:55.2379798Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-21T01:34:55.2381255Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-21T01:34:55.2382714Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-21T01:34:55.2384169Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-21T01:34:55.2385616Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-21T01:34:55.2399273Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-21T01:34:55.2477907Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-21T01:34:55.2481082Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-21T01:34:55.2483673Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-21T01:34:55.2486142Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-21T01:34:55.2488432Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-21T01:34:55.2490642Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-21T01:34:55.2492682Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-21T01:34:55.2494622Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-21T01:34:55.2496508Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-21T01:34:55.2498378Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-21T01:34:55.2550372Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-21T01:34:55.2552246Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-21T01:34:55.2553929Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-21T01:34:55.2555999Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-21T01:34:55.2562304Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-21T01:34:55.2564740Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-21T01:34:55.2567149Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-21T01:34:55.2569859Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-21T01:34:55.2572346Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-21T01:34:55.2575251Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-21T01:34:55.2578236Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-21T01:34:55.2580850Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-21T01:34:55.2583893Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-21T01:34:55.2586899Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-21T01:34:55.2590198Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-21T01:34:55.2593114Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-21T01:34:55.2596076Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-21T01:34:55.2599025Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-21T01:34:55.2602028Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-21T01:34:55.2604691Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-21T01:34:55.2607146Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-21T01:34:55.2638399Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-21T01:34:55.2686081Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-21T01:34:55.2690261Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-21T01:34:55.2743134Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-21T01:34:55.2746987Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-21T01:34:55.2751257Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-21T01:34:55.2755317Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-21T01:34:55.2759102Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-21T01:34:55.2762745Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-21T01:34:55.2766444Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-21T01:34:55.2770134Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-21T01:34:55.2773794Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-21T01:34:55.2777072Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-21T01:34:55.2780482Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-21T01:34:55.2783754Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-21T01:34:55.2786855Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-21T01:34:55.2790333Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-21T01:34:55.2793459Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-21T01:34:55.2837176Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-21T01:34:55.2910669Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-21T01:34:55.2913906Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-21T01:34:55.2916579Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-21T01:34:55.2919078Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-21T01:34:55.2921357Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-21T01:34:55.2923733Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-21T01:34:55.2926666Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-21T01:34:55.2929847Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-21T01:34:55.2932856Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-21T01:34:55.2935736Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-21T01:34:55.2938699Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-21T01:34:55.2942039Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-21T01:34:55.2944849Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-21T01:34:55.2948183Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-21T01:34:55.2950964Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-21T01:34:55.2954205Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-21T01:34:55.3083845Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-21T01:34:55.3086855Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-21T01:34:55.3090049Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-21T01:34:55.3093215Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-21T01:34:55.3096383Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-21T01:34:55.3099382Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-21T01:34:55.3102384Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-21T01:34:55.3105452Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-21T01:34:55.3109105Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-21T01:34:55.3111021Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-21T01:34:55.3112712Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-21T01:34:55.3118863Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-21T01:34:55.3184951Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-21T01:34:55.3186715Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-21T01:34:55.3189012Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-21T01:34:55.3218596Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-21T01:34:55.3220110Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-21T01:34:55.3221427Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-21T01:34:55.3222733Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-21T01:34:55.3223922Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-21T01:34:55.3224999Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-21T01:34:55.3226115Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-21T01:34:55.3315766Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-21T01:34:55.3366551Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-21T01:34:55.3367357Z creating: build/lib/ 2024-08-21T01:34:55.3472508Z inflating: build/lib/libprotobuf-lite.a 2024-08-21T01:34:55.4017026Z inflating: build/lib/libprotobuf.a 2024-08-21T01:34:55.4027890Z inflating: build/lib/libpthreadpool.a 2024-08-21T01:34:55.4037558Z inflating: build/lib/libcpuinfo.a 2024-08-21T01:34:55.4047186Z inflating: build/lib/libcpuinfo_internals.a 2024-08-21T01:34:55.4047798Z inflating: build/lib/libclog.a 2024-08-21T01:34:55.4069212Z inflating: build/lib/libnnpack.a 2024-08-21T01:34:55.4071794Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-21T01:34:55.4149161Z inflating: build/lib/libgtest.a 2024-08-21T01:34:55.4238420Z inflating: build/lib/libbenchmark.a 2024-08-21T01:34:55.4314835Z inflating: build/lib/libasmjit.a 2024-08-21T01:34:55.4323718Z inflating: build/lib/libittnotify.a 2024-08-21T01:34:55.4356915Z inflating: build/lib/libtensorpipe_uv.a 2024-08-21T01:34:55.4508575Z inflating: build/lib/libgloo.a 2024-08-21T01:34:55.4532921Z inflating: build/lib/libfmt.a 2024-08-21T01:34:55.4646411Z inflating: build/lib/libc10.so 2024-08-21T01:34:55.4648250Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-21T01:34:55.4649396Z inflating: build/lib/libtorch_global_deps.so 2024-08-21T01:34:55.5252349Z inflating: build/lib/libprotoc.a 2024-08-21T01:34:55.5274591Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-21T01:34:55.5295628Z inflating: build/lib/libgmock.a 2024-08-21T01:34:55.5296709Z inflating: build/lib/libgtest_main.a 2024-08-21T01:34:55.5297355Z inflating: build/lib/libbenchmark_main.a 2024-08-21T01:34:55.5981160Z inflating: build/lib/libtensorpipe.a 2024-08-21T01:34:56.8167344Z inflating: build/lib/libdnnl.a 2024-08-21T01:34:56.8234924Z inflating: build/lib/libc10_cuda.so 2024-08-21T01:34:56.8235492Z inflating: build/lib/libgmock_main.a 2024-08-21T01:34:56.9735513Z inflating: build/lib/libfbgemm.a 2024-08-21T01:34:57.0039586Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-21T01:34:57.0693803Z inflating: build/lib/libkineto.a 2024-08-21T01:34:57.1156855Z inflating: build/lib/libgloo_cuda.a 2024-08-21T01:34:57.1377533Z inflating: build/lib/libXNNPACK.a 2024-08-21T01:34:57.1427797Z inflating: build/lib/libonnx_proto.a 2024-08-21T01:34:57.2256544Z inflating: build/lib/libonnx.a 2024-08-21T01:35:00.1802507Z inflating: build/lib/libtorch_cpu.so 2024-08-21T01:35:00.1806615Z inflating: build/lib/libunbox_lib.a 2024-08-21T01:35:00.1811218Z inflating: build/lib/libshm.so 2024-08-21T01:35:02.7205975Z inflating: build/lib/libtorch_cuda.so 2024-08-21T01:35:02.7207059Z inflating: build/lib/libtorch.so 2024-08-21T01:35:02.7208990Z inflating: build/lib/libc10d_cuda_test.so 2024-08-21T01:35:03.7252481Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-21T01:35:03.9644778Z inflating: build/lib/libtorch_python.so 2024-08-21T01:35:03.9730020Z inflating: build/lib/libtorchbind_test.so 2024-08-21T01:35:03.9754555Z inflating: build/lib/libjitbackend_test.so 2024-08-21T01:35:03.9784666Z inflating: build/lib/libbackend_with_compiler.so 2024-08-21T01:35:03.9815573Z inflating: build/lib/libaoti_custom_ops.so 2024-08-21T01:35:03.9857086Z inflating: build/lib/libnnapi_backend.so 2024-08-21T01:35:03.9858135Z creating: build/bin/ 2024-08-21T01:35:03.9918117Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-21T01:35:03.9980103Z inflating: build/bin/c10_DeviceGuard_test 2024-08-21T01:35:04.0041536Z inflating: build/bin/c10_Device_test 2024-08-21T01:35:04.0112316Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-21T01:35:04.0176500Z inflating: build/bin/c10_Scalar_test 2024-08-21T01:35:04.0235507Z inflating: build/bin/c10_StreamGuard_test 2024-08-21T01:35:04.0296167Z inflating: build/bin/c10_SymInt_test 2024-08-21T01:35:04.0361925Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-21T01:35:04.0428537Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-21T01:35:04.0495996Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-21T01:35:04.0581220Z inflating: build/bin/c10_cow_test 2024-08-21T01:35:04.0644899Z inflating: build/bin/c10_Bitset_test 2024-08-21T01:35:04.0704040Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-21T01:35:04.0763896Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-21T01:35:04.0824032Z inflating: build/bin/c10_Half_test 2024-08-21T01:35:04.0891859Z inflating: build/bin/c10_LeftRight_test 2024-08-21T01:35:04.0958563Z inflating: build/bin/c10_Metaprogramming_test 2024-08-21T01:35:04.1019068Z inflating: build/bin/c10_Synchronized_test 2024-08-21T01:35:04.1085764Z inflating: build/bin/c10_ThreadLocal_test 2024-08-21T01:35:04.1147855Z inflating: build/bin/c10_TypeIndex_test 2024-08-21T01:35:04.1211631Z inflating: build/bin/c10_TypeList_test 2024-08-21T01:35:04.1270778Z inflating: build/bin/c10_TypeTraits_test 2024-08-21T01:35:04.1335031Z inflating: build/bin/c10_accumulate_test 2024-08-21T01:35:04.1402924Z inflating: build/bin/c10_bfloat16_test 2024-08-21T01:35:04.1464691Z inflating: build/bin/c10_bit_cast_test 2024-08-21T01:35:04.1537041Z inflating: build/bin/c10_complex_math_test 2024-08-21T01:35:04.1631127Z inflating: build/bin/c10_complex_test 2024-08-21T01:35:04.1712341Z inflating: build/bin/c10_exception_test 2024-08-21T01:35:04.1773155Z inflating: build/bin/c10_flags_test 2024-08-21T01:35:04.1833782Z inflating: build/bin/c10_generic_math_test 2024-08-21T01:35:04.2031421Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-21T01:35:04.2091980Z inflating: build/bin/c10_irange_test 2024-08-21T01:35:04.2156618Z inflating: build/bin/c10_lazy_test 2024-08-21T01:35:04.2225014Z inflating: build/bin/c10_logging_test 2024-08-21T01:35:04.2314807Z inflating: build/bin/c10_optional_test 2024-08-21T01:35:04.2389797Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-21T01:35:04.2454397Z inflating: build/bin/c10_registry_test 2024-08-21T01:35:04.2636087Z inflating: build/bin/c10_small_vector_test 2024-08-21T01:35:04.2696930Z inflating: build/bin/c10_ssize_test 2024-08-21T01:35:04.2759480Z inflating: build/bin/c10_string_util_test 2024-08-21T01:35:04.2829996Z inflating: build/bin/c10_string_view_test 2024-08-21T01:35:04.2890594Z inflating: build/bin/c10_tempfile_test 2024-08-21T01:35:04.2958472Z inflating: build/bin/c10_typeid_test 2024-08-21T01:35:04.3016743Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-21T01:35:04.3554848Z inflating: build/bin/protoc-3.13.0.0 2024-08-21T01:35:04.4091111Z inflating: build/bin/protoc 2024-08-21T01:35:04.4154412Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-21T01:35:04.4218174Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-21T01:35:04.4281237Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-21T01:35:04.4343324Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-21T01:35:04.4408507Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-21T01:35:04.4470931Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-21T01:35:04.4528970Z inflating: build/bin/c10_cuda_CUDATest 2024-08-21T01:35:04.4592705Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-21T01:35:04.4989021Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-21T01:35:04.5401702Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-21T01:35:04.5840239Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-21T01:35:04.5904832Z inflating: build/bin/BackoffTest 2024-08-21T01:35:04.5969183Z inflating: build/bin/FileStoreTest 2024-08-21T01:35:04.6038380Z inflating: build/bin/TCPStoreTest 2024-08-21T01:35:04.6103405Z inflating: build/bin/HashStoreTest 2024-08-21T01:35:04.6121158Z inflating: build/bin/ProcessGroupMPITest 2024-08-21T01:35:04.6190392Z inflating: build/bin/test_edge_op_registration 2024-08-21T01:35:04.6193831Z inflating: build/bin/torch_shm_manager 2024-08-21T01:35:04.6197155Z inflating: build/bin/example_allreduce 2024-08-21T01:35:04.6264128Z inflating: build/bin/test_dist_autograd 2024-08-21T01:35:04.6347408Z inflating: build/bin/test_cpp_rpc 2024-08-21T01:35:04.6350388Z inflating: build/bin/parallel_benchmark 2024-08-21T01:35:04.6430803Z inflating: build/bin/test_mobile_nnc 2024-08-21T01:35:04.6441571Z inflating: build/bin/aot_model_compiler_test 2024-08-21T01:35:04.6858610Z inflating: build/bin/test_lazy 2024-08-21T01:35:04.8277042Z inflating: build/bin/test_api 2024-08-21T01:35:04.8365410Z inflating: build/bin/Dict_test 2024-08-21T01:35:04.8427274Z inflating: build/bin/Dimname_test 2024-08-21T01:35:04.8504666Z inflating: build/bin/MaybeOwned_test 2024-08-21T01:35:04.8573675Z inflating: build/bin/NamedTensor_test 2024-08-21T01:35:04.8645005Z inflating: build/bin/apply_utils_test 2024-08-21T01:35:04.8715942Z inflating: build/bin/atest 2024-08-21T01:35:04.8791817Z inflating: build/bin/basic 2024-08-21T01:35:04.8858230Z inflating: build/bin/broadcast_test 2024-08-21T01:35:04.8919248Z inflating: build/bin/cpu_allocator_test 2024-08-21T01:35:04.8989391Z inflating: build/bin/cpu_generator_test 2024-08-21T01:35:04.9053232Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-21T01:35:04.9165081Z inflating: build/bin/cpu_rng_test 2024-08-21T01:35:04.9225316Z inflating: build/bin/dispatch_key_set_test 2024-08-21T01:35:04.9286177Z inflating: build/bin/dlconvertor_test 2024-08-21T01:35:04.9356482Z inflating: build/bin/extension_backend_test 2024-08-21T01:35:04.9422606Z inflating: build/bin/half_test 2024-08-21T01:35:04.9537498Z inflating: build/bin/ivalue_test 2024-08-21T01:35:04.9598213Z inflating: build/bin/lazy_tensor_test 2024-08-21T01:35:04.9663213Z inflating: build/bin/math_kernel_test 2024-08-21T01:35:04.9728199Z inflating: build/bin/memory_format_test 2024-08-21T01:35:04.9791308Z inflating: build/bin/memory_overlapping_test 2024-08-21T01:35:04.9854589Z inflating: build/bin/mobile_memory_cleanup 2024-08-21T01:35:04.9921988Z inflating: build/bin/native_test 2024-08-21T01:35:04.9982563Z inflating: build/bin/operator_name_test 2024-08-21T01:35:05.0044397Z inflating: build/bin/operators_test 2024-08-21T01:35:05.0106484Z inflating: build/bin/packedtensoraccessor_test 2024-08-21T01:35:05.0186993Z inflating: build/bin/pow_test 2024-08-21T01:35:05.0255551Z inflating: build/bin/quantized_test 2024-08-21T01:35:05.0315558Z inflating: build/bin/reduce_ops_test 2024-08-21T01:35:05.0377134Z inflating: build/bin/reportMemoryUsage_test 2024-08-21T01:35:05.0444982Z inflating: build/bin/scalar_tensor_test 2024-08-21T01:35:05.0515221Z inflating: build/bin/scalar_test 2024-08-21T01:35:05.0577513Z inflating: build/bin/StorageUtils_test 2024-08-21T01:35:05.0640496Z inflating: build/bin/stride_properties_test 2024-08-21T01:35:05.0734976Z inflating: build/bin/tensor_iterator_test 2024-08-21T01:35:05.0800067Z inflating: build/bin/test_parallel 2024-08-21T01:35:05.0803460Z inflating: build/bin/thread_init_test 2024-08-21T01:35:05.0870247Z inflating: build/bin/type_ptr_test 2024-08-21T01:35:05.0942243Z inflating: build/bin/type_test 2024-08-21T01:35:05.1004860Z inflating: build/bin/undefined_tensor_test 2024-08-21T01:35:05.1006455Z inflating: build/bin/verify_api_visibility 2024-08-21T01:35:05.1089567Z inflating: build/bin/legacy_vmap_test 2024-08-21T01:35:05.1151715Z inflating: build/bin/weakref_test 2024-08-21T01:35:05.1213660Z inflating: build/bin/wrapdim_test 2024-08-21T01:35:05.1275602Z inflating: build/bin/xla_tensor_test 2024-08-21T01:35:05.1347866Z inflating: build/bin/IListRef_test 2024-08-21T01:35:05.1475386Z inflating: build/bin/List_test 2024-08-21T01:35:05.1555307Z inflating: build/bin/KernelFunction_test 2024-08-21T01:35:05.1701388Z inflating: build/bin/kernel_function_legacy_test 2024-08-21T01:35:05.1860578Z inflating: build/bin/kernel_function_test 2024-08-21T01:35:05.2020750Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-21T01:35:05.2146017Z inflating: build/bin/kernel_lambda_test 2024-08-21T01:35:05.2219471Z inflating: build/bin/kernel_stackbased_test 2024-08-21T01:35:05.2335572Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-21T01:35:05.2396664Z inflating: build/bin/CppSignature_test 2024-08-21T01:35:05.2455155Z inflating: build/bin/op_allowlist_test 2024-08-21T01:35:05.2522048Z inflating: build/bin/backend_fallback_test 2024-08-21T01:35:05.2884692Z inflating: build/bin/op_registration_test 2024-08-21T01:35:05.2961098Z inflating: build/bin/inline_container_test 2024-08-21T01:35:05.3024516Z inflating: build/bin/cuda_apply_test 2024-08-21T01:35:05.3086858Z inflating: build/bin/cuda_allocator_test 2024-08-21T01:35:05.3152467Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-21T01:35:05.3223619Z inflating: build/bin/cuda_atomic_ops_test 2024-08-21T01:35:05.3307728Z inflating: build/bin/cuda_complex_math_test 2024-08-21T01:35:05.3378573Z inflating: build/bin/cuda_complex_test 2024-08-21T01:35:05.3438372Z inflating: build/bin/cuda_device_test 2024-08-21T01:35:05.3506883Z inflating: build/bin/cuda_cub_test 2024-08-21T01:35:05.3568547Z inflating: build/bin/cuda_dlconvertor_test 2024-08-21T01:35:05.3646574Z inflating: build/bin/cuda_distributions_test 2024-08-21T01:35:05.3714309Z inflating: build/bin/cuda_generator_test 2024-08-21T01:35:05.3773924Z inflating: build/bin/cuda_half_test 2024-08-21T01:35:05.3835088Z inflating: build/bin/cuda_integer_divider_test 2024-08-21T01:35:05.3893920Z inflating: build/bin/cuda_optional_test 2024-08-21T01:35:05.3956168Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-21T01:35:05.4019367Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-21T01:35:05.4079316Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-21T01:35:05.4151963Z inflating: build/bin/cuda_stream_test 2024-08-21T01:35:05.4211239Z inflating: build/bin/cuda_cudnn_test 2024-08-21T01:35:05.4273908Z inflating: build/bin/cuda_vectorized_test 2024-08-21T01:35:05.4291021Z inflating: build/bin/tutorial_tensorexpr 2024-08-21T01:35:05.4370255Z inflating: build/bin/ProcessGroupGlooTest 2024-08-21T01:35:05.4439419Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-21T01:35:05.4515327Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-21T01:35:05.4589870Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-21T01:35:05.5583726Z inflating: build/bin/test_tensorexpr 2024-08-21T01:35:05.6288936Z inflating: build/bin/test_jit 2024-08-21T01:35:05.6289778Z creating: .additional_ci_files/ 2024-08-21T01:35:05.6361897Z inflating: .additional_ci_files/test-times.json 2024-08-21T01:35:05.6648084Z inflating: .additional_ci_files/test-class-times.json 2024-08-21T01:35:05.6686121Z ##[group]Run rm artifacts.zip 2024-08-21T01:35:05.6686581Z rm artifacts.zip 2024-08-21T01:35:05.6700524Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:35:05.6701149Z env: 2024-08-21T01:35:05.6701467Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:05.6701999Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:05.6702593Z ##[endgroup] 2024-08-21T01:35:05.7894853Z ##[group]Run df -H 2024-08-21T01:35:05.7895247Z df -H 2024-08-21T01:35:05.7908867Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:35:05.7909498Z env: 2024-08-21T01:35:05.7909823Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:05.7910408Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:05.7911002Z ##[endgroup] 2024-08-21T01:35:05.7962862Z Filesystem Size Used Avail Use% Mounted on 2024-08-21T01:35:05.7963769Z /dev/root 1.1T 243G 823G 23% / 2024-08-21T01:35:05.7964290Z devtmpfs 45G 0 45G 0% /dev 2024-08-21T01:35:05.7964858Z tmpfs 45G 0 45G 0% /dev/shm 2024-08-21T01:35:05.7965608Z tmpfs 9.0G 1.1M 9.0G 1% /run 2024-08-21T01:35:05.7966477Z tmpfs 5.3M 0 5.3M 0% /run/lock 2024-08-21T01:35:05.7967081Z tmpfs 45G 0 45G 0% /sys/fs/cgroup 2024-08-21T01:35:05.7968225Z /dev/loop0 16M 16M 0 100% /snap/aws-cli/130 2024-08-21T01:35:05.7969265Z /dev/loop1 110M 110M 0 100% /snap/core/16928 2024-08-21T01:35:05.7970616Z /dev/loop3 59M 59M 0 100% /snap/core18/2823 2024-08-21T01:35:05.7971342Z /dev/loop2 110M 110M 0 100% /snap/core/17200 2024-08-21T01:35:05.7972023Z /dev/loop4 68M 68M 0 100% /snap/core20/2264 2024-08-21T01:35:05.7972980Z /dev/loop5 68M 68M 0 100% /snap/core20/2318 2024-08-21T01:35:05.7973647Z /dev/loop6 59M 59M 0 100% /snap/core18/2829 2024-08-21T01:35:05.7974314Z /dev/loop10 41M 41M 0 100% /snap/snapd/21465 2024-08-21T01:35:05.7974976Z /dev/loop11 97M 97M 0 100% /snap/lxd/24061 2024-08-21T01:35:05.7975631Z /dev/loop12 41M 41M 0 100% /snap/snapd/21759 2024-08-21T01:35:05.7976334Z /dev/sda15 110M 6.4M 104M 6% /boot/efi 2024-08-21T01:35:05.7977172Z /dev/loop8 371M 371M 0 100% /snap/google-cloud-sdk/468 2024-08-21T01:35:05.7977900Z /dev/loop7 97M 97M 0 100% /snap/lxd/29619 2024-08-21T01:35:05.7978708Z /dev/loop9 372M 372M 0 100% /snap/google-cloud-sdk/470 2024-08-21T01:35:05.8037377Z Prepare all required actions 2024-08-21T01:35:05.8038060Z Getting action download info 2024-08-21T01:35:05.9420159Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-21T01:35:05.9420769Z with: 2024-08-21T01:35:05.9421057Z env: 2024-08-21T01:35:05.9421371Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:05.9421919Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:05.9422493Z ##[endgroup] 2024-08-21T01:35:05.9467446Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-21T01:35:05.9468494Z with: 2024-08-21T01:35:05.9468843Z name: td_results 2024-08-21T01:35:05.9469223Z s3-bucket: gha-artifacts 2024-08-21T01:35:05.9469641Z region: us-east-1 2024-08-21T01:35:05.9469993Z env: 2024-08-21T01:35:05.9470288Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:05.9470830Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:05.9471411Z ##[endgroup] 2024-08-21T01:35:06.5575332Z (node:1151440) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-21T01:35:06.5576322Z 2024-08-21T01:35:06.5576657Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-21T01:35:06.5577924Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-21T01:35:06.5579131Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-21T01:35:06.7230791Z Found 0 objects with prefix pytorch/pytorch/10479310721/td_results/ 2024-08-21T01:35:06.7237951Z Artifact download has finished successfully 2024-08-21T01:35:06.7479011Z ##[group]Run mkdir -p .additional_ci_files 2024-08-21T01:35:06.7479655Z mkdir -p .additional_ci_files 2024-08-21T01:35:06.7480342Z mv td_results.json .additional_ci_files/td_results.json 2024-08-21T01:35:06.7493764Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:35:06.7494368Z env: 2024-08-21T01:35:06.7494690Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:06.7495258Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:06.7495885Z ##[endgroup] 2024-08-21T01:35:06.7566503Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-21T01:35:06.7584535Z ##[error]Process completed with exit code 1. 2024-08-21T01:35:06.7628831Z ##[group]Run .github/scripts/parse_ref.py 2024-08-21T01:35:06.7629416Z .github/scripts/parse_ref.py 2024-08-21T01:35:06.7641581Z shell: /usr/bin/bash -e {0} 2024-08-21T01:35:06.7641985Z env: 2024-08-21T01:35:06.7642300Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:06.7642843Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:06.7643420Z ##[endgroup] 2024-08-21T01:35:06.7969061Z Prepare all required actions 2024-08-21T01:35:06.8018772Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-21T01:35:06.8019329Z with: 2024-08-21T01:35:06.8020119Z github-token: *** 2024-08-21T01:35:06.8020472Z env: 2024-08-21T01:35:06.8020764Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:06.8021304Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:06.8021885Z ##[endgroup] 2024-08-21T01:35:06.8041424Z ##[group]Run set -eux 2024-08-21T01:35:06.8041828Z set -eux 2024-08-21T01:35:06.8042570Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-21T01:35:06.8054921Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:35:06.8055514Z env: 2024-08-21T01:35:06.8055881Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:06.8056476Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:06.8057304Z GITHUB_TOKEN: *** 2024-08-21T01:35:06.8057703Z ##[endgroup] 2024-08-21T01:35:06.8110352Z + python3 .github/scripts/get_workflow_job_id.py 10479310721 gh-ci-gcp-a100-4 2024-08-21T01:35:07.5930072Z setting job-id=29026270954 2024-08-21T01:35:07.5931199Z setting job-name=cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-08-21T01:35:07.6217508Z Prepare all required actions 2024-08-21T01:35:07.6218135Z Getting action download info 2024-08-21T01:35:07.7681634Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-21T01:35:07.7682209Z with: 2024-08-21T01:35:07.7682793Z github-token: *** 2024-08-21T01:35:07.7683723Z test-matrix: {"include": [{"config": "inductor_torchbench_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "linux.gcp.a100"}]} 2024-08-21T01:35:07.7685225Z job-name: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-08-21T01:35:07.7686191Z env: 2024-08-21T01:35:07.7686510Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:07.7687037Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:07.7687623Z ##[endgroup] 2024-08-21T01:35:07.7742296Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-21T01:35:07.7742975Z with: 2024-08-21T01:35:07.7743280Z shell: bash 2024-08-21T01:35:07.7743691Z timeout_minutes: 10 2024-08-21T01:35:07.7744059Z max_attempts: 5 2024-08-21T01:35:07.7744428Z retry_wait_seconds: 30 2024-08-21T01:35:07.7745908Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-21T01:35:07.7747840Z polling_interval_seconds: 1 2024-08-21T01:35:07.7748271Z warning_on_retry: true 2024-08-21T01:35:07.7748682Z continue_on_error: false 2024-08-21T01:35:07.7749079Z env: 2024-08-21T01:35:07.7749385Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:07.7749904Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:07.7750735Z GITHUB_TOKEN: *** 2024-08-21T01:35:07.7751092Z ##[endgroup] 2024-08-21T01:35:07.8262218Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-21T01:35:08.9880578Z Requirement already satisfied: requests==2.27.1 in /home/ubuntu/.local/lib/python3.8/site-packages (2.27.1) 2024-08-21T01:35:09.0004322Z Requirement already satisfied: pyyaml==6.0.1 in /home/ubuntu/.local/lib/python3.8/site-packages (6.0.1) 2024-08-21T01:35:09.0021699Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests==2.27.1) (1.25.8) 2024-08-21T01:35:09.0085665Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ubuntu/.local/lib/python3.8/site-packages (from requests==2.27.1) (2.0.12) 2024-08-21T01:35:09.0110928Z Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests==2.27.1) (2019.11.28) 2024-08-21T01:35:09.0126667Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /usr/lib/python3/dist-packages (from requests==2.27.1) (2.8) 2024-08-21T01:35:09.8274223Z Command completed after 1 attempt(s). 2024-08-21T01:35:09.8340914Z ##[group]Run set -x 2024-08-21T01:35:09.8341319Z set -x 2024-08-21T01:35:09.8341660Z  2024-08-21T01:35:09.8342347Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-21T01:35:09.8343188Z # in runner workspace 2024-08-21T01:35:09.8343827Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-21T01:35:09.8358451Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:35:09.8359081Z env: 2024-08-21T01:35:09.8359403Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:09.8359939Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:09.8361267Z ##[endgroup] 2024-08-21T01:35:09.8401391Z + python3 /home/weiwangmeta/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-21T01:35:09.8680507Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-21T01:35:09.8681186Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-21T01:35:09.8681754Z echo "Job name: ${JOB_NAME}" 2024-08-21T01:35:09.8682233Z  2024-08-21T01:35:09.8683211Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-21T01:35:09.8684080Z # in runner workspace 2024-08-21T01:35:09.8684788Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-21T01:35:09.8685865Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-21T01:35:09.8686404Z  --job-name "${JOB_NAME}" \ 2024-08-21T01:35:09.8687527Z  --test-matrix "{"include": [{"config": "inductor_torchbench_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "linux.gcp.a100"}]}" \ 2024-08-21T01:35:09.8688730Z  --selected-test-configs "" \ 2024-08-21T01:35:09.8689279Z  --pr-number "${PR_NUMBER}" \ 2024-08-21T01:35:09.8689780Z  --tag "${TAG}" \ 2024-08-21T01:35:09.8690228Z  --event-name "${EVENT_NAME}" \ 2024-08-21T01:35:09.8690762Z  --schedule "${SCHEDULE}" \ 2024-08-21T01:35:09.8691271Z  --branch "${HEAD_BRANCH}" 2024-08-21T01:35:09.8704966Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:35:09.8705641Z env: 2024-08-21T01:35:09.8705970Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:09.8706522Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:09.8707401Z GITHUB_TOKEN: *** 2024-08-21T01:35:09.8708689Z JOB_NAME: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-08-21T01:35:09.8709610Z PR_NUMBER: 2024-08-21T01:35:09.8709970Z TAG: ciflow/inductor/133712 2024-08-21T01:35:09.8710435Z EVENT_NAME: push 2024-08-21T01:35:09.8710793Z SCHEDULE: 2024-08-21T01:35:09.8711117Z HEAD_BRANCH: 2024-08-21T01:35:09.8711460Z ##[endgroup] 2024-08-21T01:35:09.8750085Z Workflow: inductor-periodic 2024-08-21T01:35:09.8751139Z Job name: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-08-21T01:35:10.1888952Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-21T01:35:10.4780273Z ##[group]Run echo "Filtered matrix:" 2024-08-21T01:35:10.4780840Z echo "Filtered matrix:" 2024-08-21T01:35:10.4781879Z echo "{"include": [{"config": "inductor_torchbench_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "linux.gcp.a100"}]}" 2024-08-21T01:35:10.4782923Z  2024-08-21T01:35:10.4783220Z echo 2024-08-21T01:35:10.4783659Z echo "Is the current job unstable? False" 2024-08-21T01:35:10.4784216Z  2024-08-21T01:35:10.4784520Z echo 2024-08-21T01:35:10.4784924Z echo "Is keep-going label set? False" 2024-08-21T01:35:10.4785453Z  2024-08-21T01:35:10.4785754Z echo 2024-08-21T01:35:10.4786112Z echo "Renabled issues? " 2024-08-21T01:35:10.4800021Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:35:10.4800645Z env: 2024-08-21T01:35:10.4800964Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:10.4801510Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:10.4802088Z ##[endgroup] 2024-08-21T01:35:10.4837675Z Filtered matrix: 2024-08-21T01:35:10.4838702Z {include: [{config: inductor_torchbench_smoketest_perf, shard: 1, num_shards: 1, runner: linux.gcp.a100}]} 2024-08-21T01:35:10.4840263Z 2024-08-21T01:35:10.4840609Z Is the current job unstable? False 2024-08-21T01:35:10.4841077Z 2024-08-21T01:35:10.4841472Z Is keep-going label set? False 2024-08-21T01:35:10.4841793Z 2024-08-21T01:35:10.4841923Z Renabled issues? 2024-08-21T01:35:10.4895681Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-21T01:35:10.4896596Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-21T01:35:10.4909627Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T01:35:10.4910271Z env: 2024-08-21T01:35:10.4910575Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:10.4911122Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:10.4911719Z JOB_TIMEOUT: 240 2024-08-21T01:35:10.4912055Z ##[endgroup] 2024-08-21T01:35:10.5013764Z ##[group]Run set -x 2024-08-21T01:35:10.5014240Z set -x 2024-08-21T01:35:10.5014583Z  2024-08-21T01:35:10.5014994Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-21T01:35:10.5015651Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-21T01:35:10.5016422Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-21T01:35:10.5017049Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-21T01:35:10.5017539Z else 2024-08-21T01:35:10.5017924Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-21T01:35:10.5018424Z fi 2024-08-21T01:35:10.5018730Z  2024-08-21T01:35:10.5019291Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-21T01:35:10.5020259Z # TODO: Stop building test binaries as part of the build phase 2024-08-21T01:35:10.5021098Z # Used for GPU_FLAG since that doesn't play nice 2024-08-21T01:35:10.5021830Z # shellcheck disable=SC2086,SC2090 2024-08-21T01:35:10.5022375Z container_name=$(docker run \ 2024-08-21T01:35:10.5022870Z  ${GPU_FLAG:-} \ 2024-08-21T01:35:10.5023305Z  -e BUILD_ENVIRONMENT \ 2024-08-21T01:35:10.5023781Z  -e PR_NUMBER \ 2024-08-21T01:35:10.5024395Z  -e GITHUB_ACTIONS \ 2024-08-21T01:35:10.5024858Z  -e GITHUB_REPOSITORY \ 2024-08-21T01:35:10.5025337Z  -e GITHUB_WORKFLOW \ 2024-08-21T01:35:10.5025835Z  -e GITHUB_JOB \ 2024-08-21T01:35:10.5026242Z  -e GITHUB_RUN_ID \ 2024-08-21T01:35:10.5026702Z  -e GITHUB_RUN_NUMBER \ 2024-08-21T01:35:10.5027181Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-21T01:35:10.5027999Z  -e JOB_ID \ 2024-08-21T01:35:10.5028383Z  -e JOB_NAME \ 2024-08-21T01:35:10.5028788Z  -e BASE_SHA \ 2024-08-21T01:35:10.5029184Z  -e BRANCH \ 2024-08-21T01:35:10.5029557Z  -e SHA1 \ 2024-08-21T01:35:10.5029959Z  -e AWS_DEFAULT_REGION \ 2024-08-21T01:35:10.5030435Z  -e IN_WHEEL_TEST \ 2024-08-21T01:35:10.5030881Z  -e SHARD_NUMBER \ 2024-08-21T01:35:10.5031403Z  -e TEST_CONFIG \ 2024-08-21T01:35:10.5031848Z  -e NUM_TEST_SHARDS \ 2024-08-21T01:35:10.5032324Z  -e REENABLED_ISSUES \ 2024-08-21T01:35:10.5032812Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-21T01:35:10.5033303Z  -e VERBOSE_TEST_LOGS \ 2024-08-21T01:35:10.5033775Z  -e TEST_SHOWLOCALS \ 2024-08-21T01:35:10.5034233Z  -e NO_TEST_TIMEOUT \ 2024-08-21T01:35:10.5034667Z  -e NO_TD \ 2024-08-21T01:35:10.5035048Z  -e TD_DISTRIBUTED \ 2024-08-21T01:35:10.5035498Z  -e PR_LABELS \ 2024-08-21T01:35:10.5036018Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-21T01:35:10.5036563Z  -e SCCACHE_BUCKET \ 2024-08-21T01:35:10.5037014Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-21T01:35:10.5037500Z  -e XLA_CUDA \ 2024-08-21T01:35:10.5037976Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-21T01:35:10.5038591Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-21T01:35:10.5039209Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-21T01:35:10.5039833Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-21T01:35:10.5040401Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-21T01:35:10.5040947Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-21T01:35:10.5041464Z  -e DASHBOARD_TAG \ 2024-08-21T01:35:10.5042042Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-21T01:35:10.5042719Z  --security-opt seccomp=unconfined \ 2024-08-21T01:35:10.5043265Z  --cap-add=SYS_PTRACE \ 2024-08-21T01:35:10.5043729Z  --ipc=host \ 2024-08-21T01:35:10.5044144Z  --shm-size="${SHM_SIZE}" \ 2024-08-21T01:35:10.5044607Z  --tty \ 2024-08-21T01:35:10.5044957Z  --detach \ 2024-08-21T01:35:10.5045609Z  --name="${container_name}" \ 2024-08-21T01:35:10.5046157Z  --user jenkins \ 2024-08-21T01:35:10.5046746Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-21T01:35:10.5047415Z  -w /var/lib/jenkins/workspace \ 2024-08-21T01:35:10.5047951Z  "${DOCKER_IMAGE}" 2024-08-21T01:35:10.5048358Z ) 2024-08-21T01:35:10.5048841Z # Propagate download.pytorch.org IP to container 2024-08-21T01:35:10.5050005Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-21T01:35:10.5051244Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-21T01:35:10.5052410Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-21T01:35:10.5064660Z shell: /usr/bin/bash -e {0} 2024-08-21T01:35:10.5065076Z env: 2024-08-21T01:35:10.5065375Z GIT_DEFAULT_BRANCH: main 2024-08-21T01:35:10.5065981Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:10.5066738Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-08-21T01:35:10.5067357Z PR_NUMBER: 2024-08-21T01:35:10.5068079Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-21T01:35:10.5068819Z GITHUB_WORKFLOW: inductor-periodic 2024-08-21T01:35:10.5069303Z GITHUB_JOB: test 2024-08-21T01:35:10.5069653Z GITHUB_RUN_ID: 10479310721 2024-08-21T01:35:10.5070075Z GITHUB_RUN_NUMBER: 54565 2024-08-21T01:35:10.5070485Z GITHUB_RUN_ATTEMPT: 1 2024-08-21T01:35:10.5070869Z JOB_ID: 29026270954 2024-08-21T01:35:10.5071666Z JOB_NAME: cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-08-21T01:35:10.5072566Z BRANCH: 2024-08-21T01:35:10.5072962Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:10.5073599Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:10.5074226Z TEST_CONFIG: inductor_torchbench_smoketest_perf 2024-08-21T01:35:10.5074782Z SHARD_NUMBER: 1 2024-08-21T01:35:10.5075136Z NUM_TEST_SHARDS: 1 2024-08-21T01:35:10.5075510Z REENABLED_ISSUES: 2024-08-21T01:35:10.5075943Z CONTINUE_THROUGH_ERROR: False 2024-08-21T01:35:10.5076693Z VERBOSE_TEST_LOGS: False 2024-08-21T01:35:10.5077232Z TEST_SHOWLOCALS: False 2024-08-21T01:35:10.5077728Z NO_TEST_TIMEOUT: False 2024-08-21T01:35:10.5078259Z NO_TD: False 2024-08-21T01:35:10.5078743Z TD_DISTRIBUTED: False 2024-08-21T01:35:10.5079357Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-21T01:35:10.5080082Z SCCACHE_S3_KEY_PREFIX: inductor-periodic 2024-08-21T01:35:10.5080714Z SHM_SIZE: 2g 2024-08-21T01:35:10.5082254Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:35:10.5083933Z XLA_CUDA: 2024-08-21T01:35:10.5084591Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-21T01:35:10.5109342Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-21T01:35:10.5109970Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-21T01:35:10.5110451Z DASHBOARD_TAG: 2024-08-21T01:35:10.5111080Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-21T01:35:10.5111516Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-21T01:35:10.5111955Z ##[endgroup] 2024-08-21T01:35:10.5146055Z + [[ inductor_torchbench_smoketest_perf == \m\u\l\t\i\g\p\u ]] 2024-08-21T01:35:10.5147100Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *onnx* ]] 2024-08-21T01:35:10.5148009Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-21T01:35:10.5158071Z +++ nproc --ignore=2 2024-08-21T01:35:10.5177723Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=10 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310721 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/weiwangmeta/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T01:35:14.4383280Z + container_name=cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T01:35:14.4385085Z + grep download.pytorch.org /etc/hosts 2024-08-21T01:35:14.4388686Z + docker exec -i cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-21T01:35:14.4937713Z + echo DOCKER_CONTAINER_ID=cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T01:35:14.4942319Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-21T01:35:14.4945983Z + docker exec -t cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-21T01:35:14.9765492Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-21T01:35:16.0485259Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-21T01:35:16.0491874Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-21T01:35:16.0497296Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-21T01:35:16.0501779Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-21T01:35:16.0506825Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-21T01:35:16.0516306Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.13.1) 2024-08-21T01:35:16.0538675Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-21T01:35:16.0557600Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-21T01:35:16.0580756Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.22.4) 2024-08-21T01:35:16.1051333Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-21T01:35:16.7870608Z Installing collected packages: torch 2024-08-21T01:35:28.7163697Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-21T01:35:28.7165986Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-21T01:35:28.7166925Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-21T01:35:28.8279721Z + export TERM=vt100 2024-08-21T01:35:28.8280184Z + TERM=vt100 2024-08-21T01:35:28.8283167Z ++ dirname .ci/pytorch/test.sh 2024-08-21T01:35:28.8291406Z + source .ci/pytorch/common.sh 2024-08-21T01:35:28.8296352Z +++ dirname .ci/pytorch/common.sh 2024-08-21T01:35:28.8306586Z ++ source .ci/pytorch/common_utils.sh 2024-08-21T01:35:28.8309829Z +++ declare -f -t trap_add 2024-08-21T01:35:28.8315990Z ++ set -ex 2024-08-21T01:35:28.8316892Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *rocm* ]] 2024-08-21T01:35:28.8318017Z ++ BUILD_TEST_LIBTORCH=0 2024-08-21T01:35:28.8318890Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 != *rocm* ]] 2024-08-21T01:35:28.8322781Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-21T01:35:28.8337371Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-21T01:35:28.8338098Z + trap_add cleanup_workspace EXIT 2024-08-21T01:35:28.8338785Z + trap_add_cmd=cleanup_workspace 2024-08-21T01:35:28.8339231Z + shift 2024-08-21T01:35:28.8339538Z + for trap_add_name in "$@" 2024-08-21T01:35:28.8347268Z +++ trap -p EXIT 2024-08-21T01:35:28.8350961Z ++ eval 'extract_trap_cmd ' 2024-08-21T01:35:28.8351468Z +++ extract_trap_cmd 2024-08-21T01:35:28.8352345Z +++ printf '%s\n' '' 2024-08-21T01:35:28.8352810Z ++ printf '%s\n' cleanup_workspace 2024-08-21T01:35:28.8355538Z + trap -- ' 2024-08-21T01:35:28.8355997Z cleanup_workspace' EXIT 2024-08-21T01:35:28.8356603Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-21T01:35:29.4574409Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-21T01:35:29.4599881Z + echo 'Environment variables:' 2024-08-21T01:35:29.4600404Z Environment variables: 2024-08-21T01:35:29.4600779Z + env 2024-08-21T01:35:29.4611511Z INSTALLED_DB=yes 2024-08-21T01:35:29.4612242Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-21T01:35:29.4613101Z NVIDIA_VISIBLE_DEVICES=all 2024-08-21T01:35:29.4614052Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-21T01:35:29.4615504Z GITHUB_WORKSPACE=/home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-08-21T01:35:29.4617113Z CONTINUE_THROUGH_ERROR=False 2024-08-21T01:35:29.4617896Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-21T01:35:29.4618561Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-21T01:35:29.4619981Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-08-21T01:35:29.4621013Z HOSTNAME=cb20aeb3a718 2024-08-21T01:35:29.4622393Z GITHUB_PATH=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/add_path_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.4623476Z GITHUB_ACTION=__self 2024-08-21T01:35:29.4623880Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-21T01:35:29.4629291Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-21T01:35:29.4634608Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-21T01:35:29.4635243Z NV_NVTX_VERSION=12.1.105-1 2024-08-21T01:35:29.4635667Z GITHUB_RUN_NUMBER=54565 2024-08-21T01:35:29.4636196Z TEST_CONFIG=inductor_torchbench_smoketest_perf 2024-08-21T01:35:29.4636753Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-21T01:35:29.4637343Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-21T01:35:29.4637904Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-21T01:35:29.4638476Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-21T01:35:29.4639290Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-21T01:35:29.4639778Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-21T01:35:29.4640323Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-21T01:35:29.4640915Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-21T01:35:29.4641495Z GITHUB_REF_TYPE=tag 2024-08-21T01:35:29.4641872Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-21T01:35:29.4642319Z NCCL_VERSION=2.17.1-1 2024-08-21T01:35:29.4642775Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:29.4643313Z XLA_CUDA= 2024-08-21T01:35:29.4643957Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-21T01:35:29.4646837Z *** 2024-08-21T01:35:29.4647168Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-21T01:35:29.4647647Z GITHUB_REPOSITORY_ID=65600975 2024-08-21T01:35:29.4648076Z GITHUB_ACTIONS=true 2024-08-21T01:35:29.4648461Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:29.4649104Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-21T01:35:29.4649824Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-21T01:35:29.4650410Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:29.4651041Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-21T01:35:29.4651627Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:29.4652834Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-21T01:35:29.4654099Z UCC_HOME=/usr 2024-08-21T01:35:29.4654500Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-21T01:35:29.4654977Z VERBOSE_TEST_LOGS=False 2024-08-21T01:35:29.4655366Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-21T01:35:29.4655983Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-21T01:35:29.4656593Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-21T01:35:29.4657147Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-21T01:35:29.4657595Z SHARD_NUMBER=1 2024-08-21T01:35:29.4657953Z GITHUB_REF_PROTECTED=false 2024-08-21T01:35:29.4658362Z HOME=/var/lib/jenkins 2024-08-21T01:35:29.4658777Z GITHUB_API_URL=https://api.github.com 2024-08-21T01:35:29.4659309Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-21T01:35:29.4659885Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-21T01:35:29.4660560Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-21T01:35:29.4661042Z CUDA_VERSION=12.1.1 2024-08-21T01:35:29.4661546Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-21T01:35:29.4662078Z NUM_TEST_SHARDS=1 2024-08-21T01:35:29.4662408Z UCX_HOME=/usr 2024-08-21T01:35:29.4663042Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-21T01:35:29.4664499Z GITHUB_STATE=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/save_state_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.4666179Z JOB_NAME=cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-08-21T01:35:29.4668006Z GITHUB_ENV=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/set_env_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.4669539Z GITHUB_EVENT_PATH=/home/weiwangmeta/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-21T01:35:29.4670396Z GITHUB_EVENT_NAME=push 2024-08-21T01:35:29.4670783Z DASHBOARD_TAG= 2024-08-21T01:35:29.4671133Z GITHUB_RUN_ID=10479310721 2024-08-21T01:35:29.4671680Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-21T01:35:29.4672355Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-21T01:35:29.4673696Z GITHUB_STEP_SUMMARY=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/step_summary_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.4674908Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-21T01:35:29.4675385Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-21T01:35:29.4675831Z PR_NUMBER= 2024-08-21T01:35:29.4676187Z GITHUB_RUN_ATTEMPT=1 2024-08-21T01:35:29.4676571Z ANACONDA_PYTHON_VERSION=3.10 2024-08-21T01:35:29.4677104Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-21T01:35:29.4677664Z TERM=vt100 2024-08-21T01:35:29.4678056Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-21T01:35:29.4678537Z INSTALLED_VISION=yes 2024-08-21T01:35:29.4679085Z BRANCH= 2024-08-21T01:35:29.4679421Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-21T01:35:29.4679882Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-21T01:35:29.4680394Z CUDA_PATH=/usr/local/cuda 2024-08-21T01:35:29.4681391Z GITHUB_ACTION_PATH=/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-21T01:35:29.4682380Z GITHUB_SERVER_URL=https://github.com 2024-08-21T01:35:29.4682954Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-21T01:35:29.4683520Z REENABLED_ISSUES= 2024-08-21T01:35:29.4683861Z SHLVL=1 2024-08-21T01:35:29.4684155Z MAX_JOBS=10 2024-08-21T01:35:29.4684503Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-21T01:35:29.4684916Z NVARCH=x86_64 2024-08-21T01:35:29.4685259Z GITHUB_ACTOR_ID=54816060 2024-08-21T01:35:29.4685812Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:29.4686501Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-21T01:35:29.4687099Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-21T01:35:29.4687948Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-21T01:35:29.4688650Z GITHUB_JOB=test 2024-08-21T01:35:29.4689123Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-21T01:35:29.4689810Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-21T01:35:29.4690582Z NO_TEST_TIMEOUT=False 2024-08-21T01:35:29.4690955Z TD_DISTRIBUTED=False 2024-08-21T01:35:29.4691409Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-21T01:35:29.4691943Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-21T01:35:29.4692453Z NV_NVPROF_VERSION=12.1.105-1 2024-08-21T01:35:29.4692871Z GITHUB_RETENTION_DAYS=90 2024-08-21T01:35:29.4693279Z OPENSSL_DIR=/opt/openssl 2024-08-21T01:35:29.4693688Z GITHUB_ACTION_REPOSITORY= 2024-08-21T01:35:29.4695021Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-21T01:35:29.4696443Z GITHUB_BASE_REF= 2024-08-21T01:35:29.4696828Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-21T01:35:29.4697267Z CI=true 2024-08-21T01:35:29.4697622Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-21T01:35:29.4698123Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-21T01:35:29.4698569Z JOB_ID=29026270954 2024-08-21T01:35:29.4698931Z INSTALLED_PROTOBUF=yes 2024-08-21T01:35:29.4699312Z GITHUB_HEAD_REF= 2024-08-21T01:35:29.4699660Z GITHUB_ACTION_REF= 2024-08-21T01:35:29.4700186Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-21T01:35:29.4700758Z TEST_SHOWLOCALS=False 2024-08-21T01:35:29.4701213Z GITHUB_WORKFLOW=inductor-periodic 2024-08-21T01:35:29.4701698Z DEBIAN_FRONTEND=noninteractive 2024-08-21T01:35:29.4702918Z GITHUB_OUTPUT=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/set_output_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.4704001Z NO_TD=False 2024-08-21T01:35:29.4704332Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-21T01:35:29.4704758Z _=/usr/bin/env 2024-08-21T01:35:29.4705335Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-21T01:35:29.4904210Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-21T01:35:29.4906158Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-21T01:35:29.4907339Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-21T01:35:29.4908695Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-21T01:35:29.4909457Z + BUILD_DIR=build 2024-08-21T01:35:29.4909827Z + BUILD_RENAMED_DIR=build_renamed 2024-08-21T01:35:29.4910293Z + BUILD_BIN_DIR=build/bin 2024-08-21T01:35:29.4910684Z + SHARD_NUMBER=1 2024-08-21T01:35:29.4911018Z + NUM_TEST_SHARDS=1 2024-08-21T01:35:29.4911502Z + export VALGRIND=ON 2024-08-21T01:35:29.4912095Z + VALGRIND=ON 2024-08-21T01:35:29.4912690Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *clang9* ]] 2024-08-21T01:35:29.4913277Z + [[ 0 == \1 ]] 2024-08-21T01:35:29.4913616Z + [[ False == \1 ]] 2024-08-21T01:35:29.4914513Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 != *bazel* ]] 2024-08-21T01:35:29.4915185Z ++ realpath build/custom_test_artifacts 2024-08-21T01:35:29.4925460Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-21T01:35:29.4926642Z + [[ -n '' ]] 2024-08-21T01:35:29.4927065Z + echo 'Environment variables' 2024-08-21T01:35:29.4927530Z Environment variables 2024-08-21T01:35:29.4927876Z + env 2024-08-21T01:35:29.4935665Z INSTALLED_DB=yes 2024-08-21T01:35:29.4936605Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-21T01:35:29.4937357Z NVIDIA_VISIBLE_DEVICES=all 2024-08-21T01:35:29.4938384Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-21T01:35:29.4939744Z GITHUB_WORKSPACE=/home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-08-21T01:35:29.4941115Z CONTINUE_THROUGH_ERROR=False 2024-08-21T01:35:29.4942194Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-21T01:35:29.4943332Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-21T01:35:29.4944671Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm80 2024-08-21T01:35:29.4945822Z HOSTNAME=cb20aeb3a718 2024-08-21T01:35:29.4948294Z GITHUB_PATH=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/add_path_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.4950265Z GITHUB_ACTION=__self 2024-08-21T01:35:29.4950936Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-21T01:35:29.4956926Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-21T01:35:29.4962277Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-21T01:35:29.4962909Z NV_NVTX_VERSION=12.1.105-1 2024-08-21T01:35:29.4963329Z GITHUB_RUN_NUMBER=54565 2024-08-21T01:35:29.4963798Z TEST_CONFIG=inductor_torchbench_smoketest_perf 2024-08-21T01:35:29.4964365Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-21T01:35:29.4964939Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-21T01:35:29.4965495Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-21T01:35:29.4966045Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-21T01:35:29.4966531Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-21T01:35:29.4967003Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-21T01:35:29.4967542Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-21T01:35:29.4968132Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-21T01:35:29.4968725Z GITHUB_REF_TYPE=tag 2024-08-21T01:35:29.4969104Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-21T01:35:29.4969541Z NCCL_VERSION=2.17.1-1 2024-08-21T01:35:29.4970006Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:29.4970545Z XLA_CUDA= 2024-08-21T01:35:29.4971164Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-21T01:35:29.4971632Z *** 2024-08-21T01:35:29.4971937Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-21T01:35:29.4972413Z GITHUB_REPOSITORY_ID=65600975 2024-08-21T01:35:29.4972845Z GITHUB_ACTIONS=true 2024-08-21T01:35:29.4973221Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T01:35:29.4973822Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-21T01:35:29.4974487Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-21T01:35:29.4975058Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:29.4975669Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-21T01:35:29.4976308Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:29.4977520Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-21T01:35:29.4978497Z UCC_HOME=/usr 2024-08-21T01:35:29.4979074Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-21T01:35:29.4979561Z VERBOSE_TEST_LOGS=False 2024-08-21T01:35:29.4979967Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-21T01:35:29.4980532Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-21T01:35:29.4981131Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-21T01:35:29.4981708Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-21T01:35:29.4982158Z SHARD_NUMBER=1 2024-08-21T01:35:29.4982518Z GITHUB_REF_PROTECTED=false 2024-08-21T01:35:29.4982916Z HOME=/var/lib/jenkins 2024-08-21T01:35:29.4983343Z GITHUB_API_URL=https://api.github.com 2024-08-21T01:35:29.4983868Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-21T01:35:29.4984431Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-21T01:35:29.4985111Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-21T01:35:29.4985614Z CUDA_VERSION=12.1.1 2024-08-21T01:35:29.4986162Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-21T01:35:29.4986680Z NUM_TEST_SHARDS=1 2024-08-21T01:35:29.4987034Z UCX_HOME=/usr 2024-08-21T01:35:29.4988450Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-21T01:35:29.4990974Z GITHUB_STATE=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/save_state_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.4992939Z JOB_NAME=cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) 2024-08-21T01:35:29.4994586Z GITHUB_ENV=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/set_env_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.4996207Z GITHUB_EVENT_PATH=/home/weiwangmeta/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-21T01:35:29.4997063Z GITHUB_EVENT_NAME=push 2024-08-21T01:35:29.4997436Z DASHBOARD_TAG= 2024-08-21T01:35:29.4997784Z GITHUB_RUN_ID=10479310721 2024-08-21T01:35:29.4998341Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-21T01:35:29.4999004Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-21T01:35:29.5000345Z GITHUB_STEP_SUMMARY=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/step_summary_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.5001555Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-21T01:35:29.5002045Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-21T01:35:29.5002490Z PR_NUMBER= 2024-08-21T01:35:29.5002805Z GITHUB_RUN_ATTEMPT=1 2024-08-21T01:35:29.5003169Z VALGRIND=ON 2024-08-21T01:35:29.5003511Z ANACONDA_PYTHON_VERSION=3.10 2024-08-21T01:35:29.5004033Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-21T01:35:29.5004588Z TERM=vt100 2024-08-21T01:35:29.5004974Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-21T01:35:29.5005471Z INSTALLED_VISION=yes 2024-08-21T01:35:29.5005814Z BRANCH= 2024-08-21T01:35:29.5006173Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-21T01:35:29.5006646Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-21T01:35:29.5007153Z CUDA_PATH=/usr/local/cuda 2024-08-21T01:35:29.5008130Z GITHUB_ACTION_PATH=/home/weiwangmeta/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-21T01:35:29.5009110Z GITHUB_SERVER_URL=https://github.com 2024-08-21T01:35:29.5009697Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-21T01:35:29.5010253Z REENABLED_ISSUES= 2024-08-21T01:35:29.5010591Z SHLVL=1 2024-08-21T01:35:29.5010882Z MAX_JOBS=10 2024-08-21T01:35:29.5011248Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-21T01:35:29.5011650Z NVARCH=x86_64 2024-08-21T01:35:29.5011988Z GITHUB_ACTOR_ID=54816060 2024-08-21T01:35:29.5012541Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-21T01:35:29.5013202Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-21T01:35:29.5013787Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-21T01:35:29.5014626Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-21T01:35:29.5015345Z GITHUB_JOB=test 2024-08-21T01:35:29.5015807Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-21T01:35:29.5016525Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-21T01:35:29.5017155Z NO_TEST_TIMEOUT=False 2024-08-21T01:35:29.5017679Z TD_DISTRIBUTED=False 2024-08-21T01:35:29.5018126Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-21T01:35:29.5018652Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-21T01:35:29.5019152Z NV_NVPROF_VERSION=12.1.105-1 2024-08-21T01:35:29.5019582Z GITHUB_RETENTION_DAYS=90 2024-08-21T01:35:29.5019985Z OPENSSL_DIR=/opt/openssl 2024-08-21T01:35:29.5020398Z GITHUB_ACTION_REPOSITORY= 2024-08-21T01:35:29.5021726Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-21T01:35:29.5023100Z GITHUB_BASE_REF= 2024-08-21T01:35:29.5023466Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-21T01:35:29.5023900Z CI=true 2024-08-21T01:35:29.5024265Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-21T01:35:29.5024745Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-21T01:35:29.5025186Z JOB_ID=29026270954 2024-08-21T01:35:29.5025553Z INSTALLED_PROTOBUF=yes 2024-08-21T01:35:29.5025979Z GITHUB_HEAD_REF= 2024-08-21T01:35:29.5026323Z GITHUB_ACTION_REF= 2024-08-21T01:35:29.5026848Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-21T01:35:29.5027421Z TEST_SHOWLOCALS=False 2024-08-21T01:35:29.5028284Z GITHUB_WORKFLOW=inductor-periodic 2024-08-21T01:35:29.5028759Z DEBIAN_FRONTEND=noninteractive 2024-08-21T01:35:29.5030178Z GITHUB_OUTPUT=/home/weiwangmeta/actions-runner/_work/_temp/_runner_file_commands/set_output_b1f0f23a-47b4-40bd-b61e-1e7afe32336b 2024-08-21T01:35:29.5031268Z NO_TD=False 2024-08-21T01:35:29.5031616Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-21T01:35:29.5032031Z _=/usr/bin/env 2024-08-21T01:35:29.5032428Z + echo 'Testing pytorch' 2024-08-21T01:35:29.5032827Z Testing pytorch 2024-08-21T01:35:29.5033186Z + export LANG=C.UTF-8 2024-08-21T01:35:29.5033576Z + LANG=C.UTF-8 2024-08-21T01:35:29.5033900Z + PR_NUMBER= 2024-08-21T01:35:29.5034383Z + [[ inductor_torchbench_smoketest_perf == \d\e\f\a\u\l\t ]] 2024-08-21T01:35:29.5035171Z + [[ inductor_torchbench_smoketest_perf == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-21T01:35:29.5035992Z + [[ inductor_torchbench_smoketest_perf == \s\l\o\w ]] 2024-08-21T01:35:29.5036873Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *slow-gradcheck* ]] 2024-08-21T01:35:29.5037787Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *cuda* ]] 2024-08-21T01:35:29.5038479Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-21T01:35:29.5039064Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-21T01:35:29.5039681Z + [[ inductor_torchbench_smoketest_perf == *crossref* ]] 2024-08-21T01:35:29.5040475Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *rocm* ]] 2024-08-21T01:35:29.5041258Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *xpu* ]] 2024-08-21T01:35:29.5042074Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 != *-bazel-* ]] 2024-08-21T01:35:29.5042771Z + pip_install --user ninja==1.10.2 2024-08-21T01:35:29.5043430Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-21T01:35:30.0244749Z Collecting ninja==1.10.2 2024-08-21T01:35:30.1566679Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-21T01:35:30.2053492Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-21T01:35:30.9296317Z Installing collected packages: ninja 2024-08-21T01:35:30.9391028Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-21T01:35:30.9392787Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-21T01:35:30.9447203Z Successfully installed ninja-1.10.2 2024-08-21T01:35:31.0496045Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-21T01:35:31.0499007Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-21T01:35:31.0501595Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *aarch64* ]] 2024-08-21T01:35:31.0502234Z + install_tlparse 2024-08-21T01:35:31.0502693Z + pip_install --user tlparse==0.3.25 2024-08-21T01:35:31.0503385Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-21T01:35:31.5686550Z Collecting tlparse==0.3.25 2024-08-21T01:35:31.7010275Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-21T01:35:31.7492104Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-21T01:35:32.6185801Z Installing collected packages: tlparse 2024-08-21T01:35:32.6638798Z Successfully installed tlparse-0.3.25 2024-08-21T01:35:32.7583731Z ++ python -m site --user-base 2024-08-21T01:35:32.7882515Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-21T01:35:32.7885098Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *asan* ]] 2024-08-21T01:35:32.7885987Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *-debug* ]] 2024-08-21T01:35:32.7887498Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 != *-bazel-* ]] 2024-08-21T01:35:32.7889181Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm80. Expect the assertion to pass' 2024-08-21T01:35:32.7890670Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm80. Expect the assertion to pass 2024-08-21T01:35:32.7891545Z + cd test 2024-08-21T01:35:32.7892201Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-21T01:35:34.8416945Z + [[ inductor_torchbench_smoketest_perf == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-21T01:35:34.8417917Z + [[ inductor_torchbench_smoketest_perf == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-21T01:35:34.8420340Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-21T01:35:34.8421524Z + [[ inductor_torchbench_smoketest_perf == *pr_time_benchmarks* ]] 2024-08-21T01:35:34.8422473Z + [[ inductor_torchbench_smoketest_perf == *dynamo_eager* ]] 2024-08-21T01:35:34.8423213Z + [[ inductor_torchbench_smoketest_perf == *aot_eager* ]] 2024-08-21T01:35:34.8423951Z + [[ inductor_torchbench_smoketest_perf == *aot_inductor* ]] 2024-08-21T01:35:34.8424690Z + [[ inductor_torchbench_smoketest_perf == *inductor* ]] 2024-08-21T01:35:34.8425376Z + [[ inductor_torchbench_smoketest_perf != *perf* ]] 2024-08-21T01:35:34.8426110Z + [[ inductor_torchbench_smoketest_perf == *dynamic* ]] 2024-08-21T01:35:34.8426783Z + [[ inductor_torchbench_smoketest_perf == *cpu* ]] 2024-08-21T01:35:34.8427982Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-21T01:35:34.8462862Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *libtorch* ]] 2024-08-21T01:35:34.8463818Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *-bazel-* ]] 2024-08-21T01:35:34.8466433Z + cd test 2024-08-21T01:35:34.8467012Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-21T01:35:36.7233801Z PyTorch built with: 2024-08-21T01:35:36.7234816Z - GCC 9.4 2024-08-21T01:35:36.7235553Z - C++ Version: 201703 2024-08-21T01:35:36.7237099Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-21T01:35:36.7238476Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-21T01:35:36.7239414Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-21T01:35:36.7240056Z - LAPACK is enabled (usually provided by MKL) 2024-08-21T01:35:36.7240647Z - NNPACK is enabled 2024-08-21T01:35:36.7241096Z - CPU capability usage: AVX512 2024-08-21T01:35:36.7241624Z - CUDA Runtime 12.1 2024-08-21T01:35:36.7242796Z - NVCC architecture flags: -gencode;arch=compute_80,code=sm_80 2024-08-21T01:35:36.7243701Z - CuDNN 90.1 (built against CUDA 12.4) 2024-08-21T01:35:36.7244228Z - Magma 2.6.1 2024-08-21T01:35:36.7254397Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-21T01:35:36.7263211Z 2024-08-21T01:35:37.0819766Z + cd test 2024-08-21T01:35:37.0820772Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-21T01:35:38.7815903Z ATen/Parallel: 2024-08-21T01:35:38.7816632Z at::get_num_threads() : 6 2024-08-21T01:35:38.7817140Z at::get_num_interop_threads() : 6 2024-08-21T01:35:38.7817634Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-21T01:35:38.7818095Z omp_get_max_threads() : 6 2024-08-21T01:35:38.7819463Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-21T01:35:38.7820484Z mkl_get_max_threads() : 6 2024-08-21T01:35:38.7821260Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-21T01:35:38.7822030Z std::thread::hardware_concurrency() : 12 2024-08-21T01:35:38.7822553Z Environment variables: 2024-08-21T01:35:38.7822985Z OMP_NUM_THREADS : [not set] 2024-08-21T01:35:38.7823423Z MKL_NUM_THREADS : [not set] 2024-08-21T01:35:38.7823854Z ATen parallel backend: OpenMP 2024-08-21T01:35:38.7824173Z 2024-08-21T01:35:39.0772518Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *aarch64* ]] 2024-08-21T01:35:39.0773485Z + [[ inductor_torchbench_smoketest_perf == *backward* ]] 2024-08-21T01:35:39.0774188Z + [[ inductor_torchbench_smoketest_perf == *xla* ]] 2024-08-21T01:35:39.0775168Z + [[ inductor_torchbench_smoketest_perf == *executorch* ]] 2024-08-21T01:35:39.0776386Z + [[ inductor_torchbench_smoketest_perf == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-21T01:35:39.0777631Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm80 == *libtorch* ]] 2024-08-21T01:35:39.0778421Z + [[ inductor_torchbench_smoketest_perf == distributed ]] 2024-08-21T01:35:39.0779205Z + [[ inductor_torchbench_smoketest_perf == *inductor_distributed* ]] 2024-08-21T01:35:39.0780139Z + [[ inductor_torchbench_smoketest_perf == *inductor-halide* ]] 2024-08-21T01:35:39.0781121Z + [[ inductor_torchbench_smoketest_perf == *inductor-micro-benchmark* ]] 2024-08-21T01:35:39.0781973Z + [[ inductor_torchbench_smoketest_perf == *huggingface* ]] 2024-08-21T01:35:39.0782666Z + [[ inductor_torchbench_smoketest_perf == *timm* ]] 2024-08-21T01:35:39.0783366Z + [[ inductor_torchbench_smoketest_perf == *torchbench* ]] 2024-08-21T01:35:39.0784066Z + [[ inductor_torchbench_smoketest_perf == *cpu* ]] 2024-08-21T01:35:39.0784638Z + install_torchaudio cuda 2024-08-21T01:35:39.0785028Z + local commit 2024-08-21T01:35:39.0785389Z ++ get_pinned_commit audio 2024-08-21T01:35:39.0785902Z ++ cat .github/ci_commit_pins/audio.txt 2024-08-21T01:35:39.0794680Z + commit=b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-21T01:35:39.0795307Z + [[ cuda == \c\u\d\a ]] 2024-08-21T01:35:39.0795910Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-21T01:35:39.0797135Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-21T01:35:39.0799548Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-21T01:35:39.4933865Z Collecting git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-21T01:35:39.4941128Z Cloning https://github.com/pytorch/audio.git (to revision b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862) to /tmp/pip-req-build-d3w9ozb9 2024-08-21T01:35:39.4972118Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio.git /tmp/pip-req-build-d3w9ozb9 2024-08-21T01:35:40.7686033Z Running command git rev-parse -q --verify 'sha^b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862' 2024-08-21T01:35:40.7713899Z Running command git fetch -q https://github.com/pytorch/audio.git b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-21T01:35:41.4516568Z Resolved https://github.com/pytorch/audio.git to commit b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-21T01:35:41.4518721Z Running command git submodule update --init --recursive -q 2024-08-21T01:35:44.4080501Z Preparing metadata (setup.py) ... [?25l- done 2024-08-21T01:35:44.4122876Z [?25hRequirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchaudio==2.4.0a0+b3f6f51) (2.5.0a0+git40ec5f6) 2024-08-21T01:35:44.4159754Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.13.1) 2024-08-21T01:35:44.4165321Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (4.12.2) 2024-08-21T01:35:44.4170322Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2.8.8) 2024-08-21T01:35:44.4175027Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.1.4) 2024-08-21T01:35:44.4180255Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2024.6.1) 2024-08-21T01:35:44.4188503Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (1.13.1) 2024-08-21T01:35:44.4209178Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchaudio==2.4.0a0+b3f6f51) (1.3.0) 2024-08-21T01:35:44.4813552Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchaudio==2.4.0a0+b3f6f51) (2.1.5) 2024-08-21T01:35:44.4889380Z Building wheels for collected packages: torchaudio 2024-08-21T01:39:26.7031505Z Building wheel for torchaudio (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done 2024-08-21T01:39:26.7115084Z [?25h Created wheel for torchaudio: filename=torchaudio-2.4.0a0+b3f6f51-cp310-cp310-linux_x86_64.whl size=2310393 sha256=78d297e2f100b2df8d092022c5f2dbd05c53f34de4f5b3de187753fa5c6e94b2 2024-08-21T01:39:26.7117431Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/b5/02/94/08dddd0295a3ebec4d4a4d9f226adfd865d48fb8ae4af2cc95 2024-08-21T01:39:26.7153653Z Successfully built torchaudio 2024-08-21T01:39:27.2907827Z Installing collected packages: torchaudio 2024-08-21T01:39:27.5502238Z Successfully installed torchaudio-2.4.0a0+b3f6f51 2024-08-21T01:39:27.7203675Z + install_torchtext 2024-08-21T01:39:27.7204360Z + local data_commit 2024-08-21T01:39:27.7204859Z + local text_commit 2024-08-21T01:39:27.7210066Z ++ get_pinned_commit data 2024-08-21T01:39:27.7211017Z ++ cat .github/ci_commit_pins/data.txt 2024-08-21T01:39:27.7236402Z + data_commit=11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-21T01:39:27.7240835Z ++ get_pinned_commit text 2024-08-21T01:39:27.7241439Z ++ cat .github/ci_commit_pins/text.txt 2024-08-21T01:39:27.7259820Z + text_commit=b0ebddc648d279826089db91775375221777a2db 2024-08-21T01:39:27.7261614Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-21T01:39:27.7263521Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-21T01:39:28.1580511Z Collecting git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-21T01:39:28.1587265Z Cloning https://github.com/pytorch/data.git (to revision 11bb5b847ea8b9e0d9bb82db3304daf383008d3f) to /tmp/pip-req-build-wrg0cjpt 2024-08-21T01:39:28.1650293Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/data.git /tmp/pip-req-build-wrg0cjpt 2024-08-21T01:39:29.1780149Z Running command git rev-parse -q --verify 'sha^11bb5b847ea8b9e0d9bb82db3304daf383008d3f' 2024-08-21T01:39:29.1809365Z Running command git fetch -q https://github.com/pytorch/data.git 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-21T01:39:29.6195174Z Running command git checkout -q 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-21T01:39:29.9956861Z Resolved https://github.com/pytorch/data.git to commit 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-21T01:39:29.9958819Z Running command git submodule update --init --recursive -q 2024-08-21T01:41:01.6222218Z Preparing metadata (setup.py) ... [?25l- done 2024-08-21T01:41:01.6270178Z [?25hRequirement already satisfied: urllib3>=1.25 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (1.26.19) 2024-08-21T01:41:01.6273396Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.32.3) 2024-08-21T01:41:01.6279104Z Requirement already satisfied: torch>2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.5.0a0+git40ec5f6) 2024-08-21T01:41:01.6317631Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.13.1) 2024-08-21T01:41:01.6322895Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (4.12.2) 2024-08-21T01:41:01.6326832Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.8.8) 2024-08-21T01:41:01.6331036Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.1.4) 2024-08-21T01:41:01.6335159Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2024.6.1) 2024-08-21T01:41:01.6342513Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.13.1) 2024-08-21T01:41:01.6362515Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.3.0) 2024-08-21T01:41:01.6436702Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.3.2) 2024-08-21T01:41:01.6441576Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.7) 2024-08-21T01:41:01.6448488Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (2024.7.4) 2024-08-21T01:41:01.7080752Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.1.5) 2024-08-21T01:41:01.7161805Z Building wheels for collected packages: torchdata 2024-08-21T01:41:04.2355129Z Building wheel for torchdata (setup.py) ... [?25l- \ | done 2024-08-21T01:41:04.2367621Z [?25h Created wheel for torchdata: filename=torchdata-0.7.0a0+11bb5b8-py3-none-any.whl size=182970 sha256=0b284cd7aaf48b519d87215c4604319c04ca1a806ca6625d46dda235ef13a739 2024-08-21T01:41:04.2369720Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/ca/59/a3/c8250bfc8d3d4d639498d4beb2e0f0e70b9a508ac61fde85ce 2024-08-21T01:41:04.2407334Z Successfully built torchdata 2024-08-21T01:41:04.8133791Z Installing collected packages: torchdata 2024-08-21T01:41:04.9245467Z Successfully installed torchdata-0.7.0a0+11bb5b8 2024-08-21T01:41:07.9049036Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-21T01:41:07.9051016Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-21T01:41:08.3257789Z Collecting git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-21T01:41:08.3265222Z Cloning https://github.com/pytorch/text.git (to revision b0ebddc648d279826089db91775375221777a2db) to /tmp/pip-req-build-shv91z93 2024-08-21T01:41:08.3293822Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/text.git /tmp/pip-req-build-shv91z93 2024-08-21T01:41:09.6323984Z Running command git rev-parse -q --verify 'sha^b0ebddc648d279826089db91775375221777a2db' 2024-08-21T01:41:09.6352163Z Running command git fetch -q https://github.com/pytorch/text.git b0ebddc648d279826089db91775375221777a2db 2024-08-21T01:41:10.4115270Z Running command git checkout -q b0ebddc648d279826089db91775375221777a2db 2024-08-21T01:41:10.8005331Z Resolved https://github.com/pytorch/text.git to commit b0ebddc648d279826089db91775375221777a2db 2024-08-21T01:41:10.8007774Z Running command git submodule update --init --recursive -q 2024-08-21T01:41:19.9880187Z Preparing metadata (setup.py) ... [?25l- done 2024-08-21T01:41:19.9927556Z [?25hRequirement already satisfied: tqdm in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (4.66.5) 2024-08-21T01:41:19.9932584Z Requirement already satisfied: requests in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.32.3) 2024-08-21T01:41:19.9939943Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.5.0a0+git40ec5f6) 2024-08-21T01:41:19.9944749Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (1.22.4) 2024-08-21T01:41:19.9952639Z Requirement already satisfied: torchdata in /var/lib/jenkins/.local/lib/python3.10/site-packages (from torchtext==0.16.0a0+b0ebddc) (0.7.0a0+11bb5b8) 2024-08-21T01:41:19.9988449Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.3.2) 2024-08-21T01:41:19.9995356Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.7) 2024-08-21T01:41:20.0003292Z Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (1.26.19) 2024-08-21T01:41:20.0010647Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (2024.7.4) 2024-08-21T01:41:20.0048673Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.13.1) 2024-08-21T01:41:20.0053948Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (4.12.2) 2024-08-21T01:41:20.0060585Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2.8.8) 2024-08-21T01:41:20.0068026Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.1.4) 2024-08-21T01:41:20.0074025Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2024.6.1) 2024-08-21T01:41:20.0084511Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (1.13.1) 2024-08-21T01:41:20.0107203Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchtext==0.16.0a0+b0ebddc) (1.3.0) 2024-08-21T01:41:20.0879900Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchtext==0.16.0a0+b0ebddc) (2.1.5) 2024-08-21T01:41:20.0968482Z Building wheels for collected packages: torchtext 2024-08-21T01:42:13.9313507Z Building wheel for torchtext (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-08-21T01:42:13.9390653Z [?25h Created wheel for torchtext: filename=torchtext-0.16.0a0+b0ebddc-cp310-cp310-linux_x86_64.whl size=2045467 sha256=cd4d65903854c741e9b202a761f1a65550c35251ebb42984f3f0f465c4603bb2 2024-08-21T01:42:13.9392897Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/e5/3d/d7/faafad098ec9437ecdf4495c0f5e72b817fffbb06063e32cbf 2024-08-21T01:42:13.9424925Z Successfully built torchtext 2024-08-21T01:42:14.5154457Z Installing collected packages: torchtext 2024-08-21T01:42:14.6512262Z Successfully installed torchtext-0.16.0a0+b0ebddc 2024-08-21T01:42:14.8154006Z + install_torchvision 2024-08-21T01:42:14.8154481Z + local orig_preload 2024-08-21T01:42:14.8154855Z + local commit 2024-08-21T01:42:14.8160276Z ++ get_pinned_commit vision 2024-08-21T01:42:14.8160965Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-21T01:42:14.8179805Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-21T01:42:14.8180542Z + orig_preload= 2024-08-21T01:42:14.8181299Z + '[' -n '' ']' 2024-08-21T01:42:14.8182737Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-21T01:42:14.8184654Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-21T01:42:15.2471173Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-21T01:42:15.2479673Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-gxt18j_w 2024-08-21T01:42:15.2510908Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-gxt18j_w 2024-08-21T01:42:17.4203665Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-21T01:42:17.4231867Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-21T01:42:19.3243520Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-21T01:42:19.8546775Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-21T01:42:23.0838256Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-21T01:42:23.0882994Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2024-08-21T01:42:23.0888811Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-21T01:42:23.0894981Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-21T01:42:23.0982572Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-21T01:42:23.0989639Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-21T01:42:23.0993876Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-21T01:42:23.0998645Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-21T01:42:23.1003085Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-21T01:42:23.1010629Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-21T01:42:23.1030231Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-21T01:42:23.1630846Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-21T01:42:23.1712472Z Building wheels for collected packages: torchvision 2024-08-21T01:44:16.4358840Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-08-21T01:44:16.4437442Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2115942 sha256=67f34e4f0597a27b3369d3c0f1fcab16ed3c727c72e609735f0bba93ae030447 2024-08-21T01:44:16.4439969Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-21T01:44:16.4484924Z Successfully built torchvision 2024-08-21T01:44:17.0274645Z Installing collected packages: torchvision 2024-08-21T01:44:17.5409112Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-21T01:44:17.6853753Z + '[' -n '' ']' 2024-08-21T01:44:17.6854301Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-21T01:44:17.6855041Z + pip_install git+https://github.com/pytorch/ao.git 2024-08-21T01:44:17.6876570Z + pip install --progress-bar off git+https://github.com/pytorch/ao.git 2024-08-21T01:44:18.1089768Z Collecting git+https://github.com/pytorch/ao.git 2024-08-21T01:44:18.1096136Z Cloning https://github.com/pytorch/ao.git to /tmp/pip-req-build-_m1xip2g 2024-08-21T01:44:18.1127016Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/ao.git /tmp/pip-req-build-_m1xip2g 2024-08-21T01:44:19.2119539Z Resolved https://github.com/pytorch/ao.git to commit 227d4bfece6bb00f0365642ad1d7afd509079ba4 2024-08-21T01:44:22.3208488Z Preparing metadata (setup.py) ... [?25l- done 2024-08-21T01:44:22.3254845Z [?25hBuilding wheels for collected packages: torchao 2024-08-21T01:46:05.6437673Z Building wheel for torchao (setup.py) ... [?25l- \ | / - \ | / done 2024-08-21T01:46:05.6476729Z [?25h Created wheel for torchao: filename=torchao-0.4.0+git227d4bf-cp310-cp310-linux_x86_64.whl size=928019 sha256=33c821bbe91733f40dc02b1642fc8e0a7197ca75b81da1d4f8abc1aecbadbb28 2024-08-21T01:46:05.6480967Z Stored in directory: /tmp/pip-ephem-wheel-cache-fcwewcc0/wheels/c0/df/d8/b1e2f5f1ea3bf141268a83a667137647b234f04c636e1e7a5e 2024-08-21T01:46:05.6520725Z Successfully built torchao 2024-08-21T01:46:06.2925265Z Installing collected packages: torchao 2024-08-21T01:46:06.5798474Z Successfully installed torchao-0.4.0+git227d4bf 2024-08-21T01:46:06.6988072Z + id=0 2024-08-21T01:46:06.6988932Z + pip_install opencv-python==4.8.0.74 2024-08-21T01:46:06.6989703Z + pip install --progress-bar off opencv-python==4.8.0.74 2024-08-21T01:46:07.7675407Z Collecting opencv-python==4.8.0.74 2024-08-21T01:46:07.9096260Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB) 2024-08-21T01:46:07.9250916Z Requirement already satisfied: numpy>=1.21.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opencv-python==4.8.0.74) (1.22.4) 2024-08-21T01:46:07.9666052Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB) 2024-08-21T01:46:09.9056603Z Installing collected packages: opencv-python 2024-08-21T01:46:11.0665139Z Successfully installed opencv-python-4.8.0.74 2024-08-21T01:46:11.1844415Z + [[ inductor_torchbench_smoketest_perf == *inductor_torchbench_smoketest_perf* ]] 2024-08-21T01:46:11.1845562Z + checkout_install_torchbench hf_Bert hf_Albert nanogpt timm_vision_transformer 2024-08-21T01:46:11.1846383Z + local commit 2024-08-21T01:46:11.1850118Z ++ get_pinned_commit torchbench 2024-08-21T01:46:11.1851164Z ++ cat .github/ci_commit_pins/torchbench.txt 2024-08-21T01:46:11.1873359Z + commit=23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-21T01:46:11.1874641Z + git clone https://github.com/pytorch/benchmark torchbench 2024-08-21T01:46:11.1895184Z Cloning into 'torchbench'... 2024-08-21T01:46:11.5399909Z remote: Enumerating objects: 31043, done. 2024-08-21T01:46:11.5400687Z remote: Counting objects: 0% (1/1783) 2024-08-21T01:46:11.5401529Z remote: Counting objects: 1% (18/1783) 2024-08-21T01:46:11.5402715Z remote: Counting objects: 2% (36/1783) 2024-08-21T01:46:11.5406305Z remote: Counting objects: 3% (54/1783) 2024-08-21T01:46:11.5407362Z remote: Counting objects: 4% (72/1783) 2024-08-21T01:46:11.5408475Z remote: Counting objects: 5% (90/1783) 2024-08-21T01:46:11.5409516Z remote: Counting objects: 6% (107/1783) 2024-08-21T01:46:11.5410754Z remote: Counting objects: 7% (125/1783) 2024-08-21T01:46:11.5411840Z remote: Counting objects: 8% (143/1783) 2024-08-21T01:46:11.5413018Z remote: Counting objects: 9% (161/1783) 2024-08-21T01:46:11.5414011Z remote: Counting objects: 10% (179/1783) 2024-08-21T01:46:11.5415165Z remote: Counting objects: 11% (197/1783) 2024-08-21T01:46:11.5416470Z remote: Counting objects: 12% (214/1783) 2024-08-21T01:46:11.5417643Z remote: Counting objects: 13% (232/1783) 2024-08-21T01:46:11.5418281Z remote: Counting objects: 14% (250/1783) 2024-08-21T01:46:11.5419382Z remote: Counting objects: 15% (268/1783) 2024-08-21T01:46:11.5420636Z remote: Counting objects: 16% (286/1783) 2024-08-21T01:46:11.5421817Z remote: Counting objects: 17% (304/1783) 2024-08-21T01:46:11.5422956Z remote: Counting objects: 18% (321/1783) 2024-08-21T01:46:11.5424121Z remote: Counting objects: 19% (339/1783) 2024-08-21T01:46:11.5425360Z remote: Counting objects: 20% (357/1783) 2024-08-21T01:46:11.5426688Z remote: Counting objects: 21% (375/1783) 2024-08-21T01:46:11.5428150Z remote: Counting objects: 22% (393/1783) 2024-08-21T01:46:11.5429261Z remote: Counting objects: 23% (411/1783) 2024-08-21T01:46:11.5429908Z remote: Counting objects: 24% (428/1783) 2024-08-21T01:46:11.5430543Z remote: Counting objects: 25% (446/1783) 2024-08-21T01:46:11.5431165Z remote: Counting objects: 26% (464/1783) 2024-08-21T01:46:11.5432230Z remote: Counting objects: 27% (482/1783) 2024-08-21T01:46:11.5432884Z remote: Counting objects: 28% (500/1783) 2024-08-21T01:46:11.5433519Z remote: Counting objects: 29% (518/1783) 2024-08-21T01:46:11.5434148Z remote: Counting objects: 30% (535/1783) 2024-08-21T01:46:11.5434843Z remote: Counting objects: 31% (553/1783) 2024-08-21T01:46:11.5435491Z remote: Counting objects: 32% (571/1783) 2024-08-21T01:46:11.5436158Z remote: Counting objects: 33% (589/1783) 2024-08-21T01:46:11.5436795Z remote: Counting objects: 34% (607/1783) 2024-08-21T01:46:11.5437430Z remote: Counting objects: 35% (625/1783) 2024-08-21T01:46:11.5438059Z remote: Counting objects: 36% (642/1783) 2024-08-21T01:46:11.5438738Z remote: Counting objects: 37% (660/1783) 2024-08-21T01:46:11.5439372Z remote: Counting objects: 38% (678/1783) 2024-08-21T01:46:11.5440013Z remote: Counting objects: 39% (696/1783) 2024-08-21T01:46:11.5440649Z remote: Counting objects: 40% (714/1783) 2024-08-21T01:46:11.5441276Z remote: Counting objects: 41% (732/1783) 2024-08-21T01:46:11.5441906Z remote: Counting objects: 42% (749/1783) 2024-08-21T01:46:11.5442535Z remote: Counting objects: 43% (767/1783) 2024-08-21T01:46:11.5443166Z remote: Counting objects: 44% (785/1783) 2024-08-21T01:46:11.5443984Z remote: Counting objects: 45% (803/1783) 2024-08-21T01:46:11.5444655Z remote: Counting objects: 46% (821/1783) 2024-08-21T01:46:11.5445322Z remote: Counting objects: 47% (839/1783) 2024-08-21T01:46:11.5445996Z remote: Counting objects: 48% (856/1783) 2024-08-21T01:46:11.5446617Z remote: Counting objects: 49% (874/1783) 2024-08-21T01:46:11.5447290Z remote: Counting objects: 50% (892/1783) 2024-08-21T01:46:11.5447970Z remote: Counting objects: 51% (910/1783) 2024-08-21T01:46:11.5448966Z remote: Counting objects: 52% (928/1783) 2024-08-21T01:46:11.5449717Z remote: Counting objects: 53% (945/1783) 2024-08-21T01:46:11.5450358Z remote: Counting objects: 54% (963/1783) 2024-08-21T01:46:11.5451002Z remote: Counting objects: 55% (981/1783) 2024-08-21T01:46:11.5451680Z remote: Counting objects: 56% (999/1783) 2024-08-21T01:46:11.5452341Z remote: Counting objects: 57% (1017/1783) 2024-08-21T01:46:11.5453001Z remote: Counting objects: 58% (1035/1783) 2024-08-21T01:46:11.5453658Z remote: Counting objects: 59% (1052/1783) 2024-08-21T01:46:11.5454308Z remote: Counting objects: 60% (1070/1783) 2024-08-21T01:46:11.5454966Z remote: Counting objects: 61% (1088/1783) 2024-08-21T01:46:11.5455608Z remote: Counting objects: 62% (1106/1783) 2024-08-21T01:46:11.5456289Z remote: Counting objects: 63% (1124/1783) 2024-08-21T01:46:11.5456951Z remote: Counting objects: 64% (1142/1783) 2024-08-21T01:46:11.5457625Z remote: Counting objects: 65% (1159/1783) 2024-08-21T01:46:11.5458271Z remote: Counting objects: 66% (1177/1783) 2024-08-21T01:46:11.5458936Z remote: Counting objects: 67% (1195/1783) 2024-08-21T01:46:11.5459564Z remote: Counting objects: 68% (1213/1783) 2024-08-21T01:46:11.5460213Z remote: Counting objects: 69% (1231/1783) 2024-08-21T01:46:11.5460920Z remote: Counting objects: 70% (1249/1783) 2024-08-21T01:46:11.5461562Z remote: Counting objects: 71% (1266/1783) 2024-08-21T01:46:11.5462220Z remote: Counting objects: 72% (1284/1783) 2024-08-21T01:46:11.5462870Z remote: Counting objects: 73% (1302/1783) 2024-08-21T01:46:11.5463511Z remote: Counting objects: 74% (1320/1783) 2024-08-21T01:46:11.5464152Z remote: Counting objects: 75% (1338/1783) 2024-08-21T01:46:11.5464782Z remote: Counting objects: 76% (1356/1783) 2024-08-21T01:46:11.5465422Z remote: Counting objects: 77% (1373/1783) 2024-08-21T01:46:11.5466101Z remote: Counting objects: 78% (1391/1783) 2024-08-21T01:46:11.5466737Z remote: Counting objects: 79% (1409/1783) 2024-08-21T01:46:11.5467389Z remote: Counting objects: 80% (1427/1783) 2024-08-21T01:46:11.5468333Z remote: Counting objects: 81% (1445/1783) 2024-08-21T01:46:11.5469226Z remote: Counting objects: 82% (1463/1783) 2024-08-21T01:46:11.5469865Z remote: Counting objects: 83% (1480/1783) 2024-08-21T01:46:11.5470513Z remote: Counting objects: 84% (1498/1783) 2024-08-21T01:46:11.5471195Z remote: Counting objects: 85% (1516/1783) 2024-08-21T01:46:11.5471839Z remote: Counting objects: 86% (1534/1783) 2024-08-21T01:46:11.5472477Z remote: Counting objects: 87% (1552/1783) 2024-08-21T01:46:11.5473150Z remote: Counting objects: 88% (1570/1783) 2024-08-21T01:46:11.5473784Z remote: Counting objects: 89% (1587/1783) 2024-08-21T01:46:11.5474420Z remote: Counting objects: 90% (1605/1783) 2024-08-21T01:46:11.5475048Z remote: Counting objects: 91% (1623/1783) 2024-08-21T01:46:11.5475689Z remote: Counting objects: 92% (1641/1783) 2024-08-21T01:46:11.5476387Z remote: Counting objects: 93% (1659/1783) 2024-08-21T01:46:11.5477029Z remote: Counting objects: 94% (1677/1783) 2024-08-21T01:46:11.5477657Z remote: Counting objects: 95% (1694/1783) 2024-08-21T01:46:11.5478313Z remote: Counting objects: 96% (1712/1783) 2024-08-21T01:46:11.5478960Z remote: Counting objects: 97% (1730/1783) 2024-08-21T01:46:11.5479603Z remote: Counting objects: 98% (1748/1783) 2024-08-21T01:46:11.5480227Z remote: Counting objects: 99% (1766/1783) 2024-08-21T01:46:11.5481048Z remote: Counting objects: 100% (1783/1783) 2024-08-21T01:46:11.5481743Z remote: Counting objects: 100% (1783/1783), done. 2024-08-21T01:46:11.5482443Z remote: Compressing objects: 0% (1/929) 2024-08-21T01:46:11.5483105Z remote: Compressing objects: 1% (10/929) 2024-08-21T01:46:11.5586166Z remote: Compressing objects: 2% (19/929) 2024-08-21T01:46:11.5691517Z remote: Compressing objects: 3% (28/929) 2024-08-21T01:46:11.5746820Z remote: Compressing objects: 4% (38/929) 2024-08-21T01:46:11.5799601Z remote: Compressing objects: 5% (47/929) 2024-08-21T01:46:11.5841454Z remote: Compressing objects: 6% (56/929) 2024-08-21T01:46:11.5872537Z remote: Compressing objects: 7% (66/929) 2024-08-21T01:46:11.5903089Z remote: Compressing objects: 8% (75/929) 2024-08-21T01:46:11.5940712Z remote: Compressing objects: 9% (84/929) 2024-08-21T01:46:11.5969853Z remote: Compressing objects: 10% (93/929) 2024-08-21T01:46:11.5986014Z remote: Compressing objects: 11% (103/929) 2024-08-21T01:46:11.5997571Z remote: Compressing objects: 12% (112/929) 2024-08-21T01:46:11.6014117Z remote: Compressing objects: 13% (121/929) 2024-08-21T01:46:11.6029832Z remote: Compressing objects: 14% (131/929) 2024-08-21T01:46:11.6038316Z remote: Compressing objects: 15% (140/929) 2024-08-21T01:46:11.6046528Z remote: Compressing objects: 16% (149/929) 2024-08-21T01:46:11.6052950Z remote: Compressing objects: 17% (158/929) 2024-08-21T01:46:11.6056088Z remote: Compressing objects: 18% (168/929) 2024-08-21T01:46:11.6059419Z remote: Compressing objects: 19% (177/929) 2024-08-21T01:46:11.6061118Z remote: Compressing objects: 20% (186/929) 2024-08-21T01:46:11.6062429Z remote: Compressing objects: 21% (196/929) 2024-08-21T01:46:11.6063491Z remote: Compressing objects: 22% (205/929) 2024-08-21T01:46:11.6065549Z remote: Compressing objects: 23% (214/929) 2024-08-21T01:46:11.6067289Z remote: Compressing objects: 24% (223/929) 2024-08-21T01:46:11.6068573Z remote: Compressing objects: 25% (233/929) 2024-08-21T01:46:11.6069244Z remote: Compressing objects: 26% (242/929) 2024-08-21T01:46:11.6072815Z remote: Compressing objects: 27% (251/929) 2024-08-21T01:46:11.6079410Z remote: Compressing objects: 28% (261/929) 2024-08-21T01:46:11.6080593Z remote: Compressing objects: 29% (270/929) 2024-08-21T01:46:11.6084172Z remote: Compressing objects: 30% (279/929) 2024-08-21T01:46:11.6088058Z remote: Compressing objects: 31% (288/929) 2024-08-21T01:46:11.6091019Z remote: Compressing objects: 32% (298/929) 2024-08-21T01:46:11.6093148Z remote: Compressing objects: 33% (307/929) 2024-08-21T01:46:11.6098934Z remote: Compressing objects: 34% (316/929) 2024-08-21T01:46:11.6102393Z remote: Compressing objects: 35% (326/929) 2024-08-21T01:46:11.6103519Z remote: Compressing objects: 36% (335/929) 2024-08-21T01:46:11.6104355Z remote: Compressing objects: 37% (344/929) 2024-08-21T01:46:11.6107139Z remote: Compressing objects: 38% (354/929) 2024-08-21T01:46:11.6108678Z remote: Compressing objects: 39% (363/929) 2024-08-21T01:46:11.6110238Z remote: Compressing objects: 40% (372/929) 2024-08-21T01:46:11.6111461Z remote: Compressing objects: 41% (381/929) 2024-08-21T01:46:11.6116229Z remote: Compressing objects: 42% (391/929) 2024-08-21T01:46:11.6117399Z remote: Compressing objects: 43% (400/929) 2024-08-21T01:46:11.6118686Z remote: Compressing objects: 44% (409/929) 2024-08-21T01:46:11.6119954Z remote: Compressing objects: 45% (419/929) 2024-08-21T01:46:11.6121034Z remote: Compressing objects: 46% (428/929) 2024-08-21T01:46:11.6122275Z remote: Compressing objects: 47% (437/929) 2024-08-21T01:46:11.6123594Z remote: Compressing objects: 48% (446/929) 2024-08-21T01:46:11.6124967Z remote: Compressing objects: 49% (456/929) 2024-08-21T01:46:11.6125633Z remote: Compressing objects: 50% (465/929) 2024-08-21T01:46:11.6126304Z remote: Compressing objects: 51% (474/929) 2024-08-21T01:46:11.6127319Z remote: Compressing objects: 52% (484/929) 2024-08-21T01:46:11.6128160Z remote: Compressing objects: 53% (493/929) 2024-08-21T01:46:11.6128892Z remote: Compressing objects: 54% (502/929) 2024-08-21T01:46:11.6129537Z remote: Compressing objects: 55% (511/929) 2024-08-21T01:46:11.6130194Z remote: Compressing objects: 56% (521/929) 2024-08-21T01:46:11.6130846Z remote: Compressing objects: 57% (530/929) 2024-08-21T01:46:11.6131499Z remote: Compressing objects: 58% (539/929) 2024-08-21T01:46:11.6132136Z remote: Compressing objects: 59% (549/929) 2024-08-21T01:46:11.6132791Z remote: Compressing objects: 60% (558/929) 2024-08-21T01:46:11.6133460Z remote: Compressing objects: 61% (567/929) 2024-08-21T01:46:11.6134116Z remote: Compressing objects: 62% (576/929) 2024-08-21T01:46:11.6134757Z remote: Compressing objects: 63% (586/929) 2024-08-21T01:46:11.6135408Z remote: Compressing objects: 64% (595/929) 2024-08-21T01:46:11.6136100Z remote: Compressing objects: 65% (604/929) 2024-08-21T01:46:11.6136758Z remote: Compressing objects: 66% (614/929) 2024-08-21T01:46:11.6137397Z remote: Compressing objects: 67% (623/929) 2024-08-21T01:46:11.6138059Z remote: Compressing objects: 68% (632/929) 2024-08-21T01:46:11.6139059Z remote: Compressing objects: 69% (642/929) 2024-08-21T01:46:11.6139723Z remote: Compressing objects: 70% (651/929) 2024-08-21T01:46:11.6140367Z remote: Compressing objects: 71% (660/929) 2024-08-21T01:46:11.6141055Z remote: Compressing objects: 72% (669/929) 2024-08-21T01:46:11.6141707Z remote: Compressing objects: 73% (679/929) 2024-08-21T01:46:11.6142393Z remote: Compressing objects: 74% (688/929) 2024-08-21T01:46:11.6143042Z remote: Compressing objects: 75% (697/929) 2024-08-21T01:46:11.6144116Z remote: Compressing objects: 76% (707/929) 2024-08-21T01:46:11.6149914Z remote: Compressing objects: 77% (716/929) 2024-08-21T01:46:11.6154892Z remote: Compressing objects: 78% (725/929) 2024-08-21T01:46:11.6162570Z remote: Compressing objects: 79% (734/929) 2024-08-21T01:46:11.6168722Z remote: Compressing objects: 80% (744/929) 2024-08-21T01:46:11.6178068Z remote: Compressing objects: 81% (753/929) 2024-08-21T01:46:11.6187669Z remote: Compressing objects: 82% (762/929) 2024-08-21T01:46:11.6191326Z remote: Compressing objects: 83% (772/929) 2024-08-21T01:46:11.6193251Z remote: Compressing objects: 84% (781/929) 2024-08-21T01:46:11.6198098Z remote: Compressing objects: 85% (790/929) 2024-08-21T01:46:11.6203155Z remote: Compressing objects: 86% (799/929) 2024-08-21T01:46:11.6207770Z remote: Compressing objects: 87% (809/929) 2024-08-21T01:46:11.6211815Z remote: Compressing objects: 88% (818/929) 2024-08-21T01:46:11.6215351Z remote: Compressing objects: 89% (827/929) 2024-08-21T01:46:11.6218135Z remote: Compressing objects: 90% (837/929) 2024-08-21T01:46:11.6222279Z remote: Compressing objects: 91% (846/929) 2024-08-21T01:46:11.6226265Z remote: Compressing objects: 92% (855/929) 2024-08-21T01:46:11.6230183Z remote: Compressing objects: 93% (864/929) 2024-08-21T01:46:11.6232272Z remote: Compressing objects: 94% (874/929) 2024-08-21T01:46:11.6234135Z remote: Compressing objects: 95% (883/929) 2024-08-21T01:46:11.6238174Z remote: Compressing objects: 96% (892/929) 2024-08-21T01:46:11.6240514Z remote: Compressing objects: 97% (902/929) 2024-08-21T01:46:11.6241346Z remote: Compressing objects: 98% (911/929) 2024-08-21T01:46:11.6242318Z remote: Compressing objects: 99% (920/929) 2024-08-21T01:46:11.6243320Z remote: Compressing objects: 100% (929/929) 2024-08-21T01:46:11.6244065Z remote: Compressing objects: 100% (929/929), done. 2024-08-21T01:46:11.6419881Z Receiving objects: 0% (1/31043) 2024-08-21T01:46:11.6563070Z Receiving objects: 1% (311/31043) 2024-08-21T01:46:11.6714155Z Receiving objects: 2% (621/31043) 2024-08-21T01:46:11.6855459Z Receiving objects: 3% (932/31043) 2024-08-21T01:46:11.6995130Z Receiving objects: 4% (1242/31043) 2024-08-21T01:46:11.7091482Z Receiving objects: 5% (1553/31043) 2024-08-21T01:46:11.7236315Z Receiving objects: 6% (1863/31043) 2024-08-21T01:46:11.7318565Z Receiving objects: 7% (2174/31043) 2024-08-21T01:46:11.7448671Z Receiving objects: 8% (2484/31043) 2024-08-21T01:46:11.7594635Z Receiving objects: 9% (2794/31043) 2024-08-21T01:46:11.7848035Z Receiving objects: 10% (3105/31043) 2024-08-21T01:46:11.7990056Z Receiving objects: 11% (3415/31043) 2024-08-21T01:46:11.8187184Z Receiving objects: 12% (3726/31043) 2024-08-21T01:46:11.8552335Z Receiving objects: 13% (4036/31043) 2024-08-21T01:46:11.9146741Z Receiving objects: 14% (4347/31043) 2024-08-21T01:46:11.9370974Z Receiving objects: 15% (4657/31043) 2024-08-21T01:46:11.9564191Z Receiving objects: 16% (4967/31043) 2024-08-21T01:46:11.9688992Z Receiving objects: 17% (5278/31043) 2024-08-21T01:46:11.9754902Z Receiving objects: 18% (5588/31043) 2024-08-21T01:46:11.9947349Z Receiving objects: 19% (5899/31043) 2024-08-21T01:46:11.9987018Z Receiving objects: 20% (6209/31043) 2024-08-21T01:46:12.0023591Z Receiving objects: 21% (6520/31043) 2024-08-21T01:46:12.0244401Z Receiving objects: 22% (6830/31043) 2024-08-21T01:46:12.6279198Z Receiving objects: 23% (7140/31043) 2024-08-21T01:46:13.6305223Z Receiving objects: 23% (7248/31043), 5.86 MiB | 11.67 MiB/s 2024-08-21T01:46:13.6629080Z Receiving objects: 23% (7404/31043), 82.40 MiB | 41.18 MiB/s 2024-08-21T01:46:14.2782035Z Receiving objects: 24% (7451/31043), 82.40 MiB | 41.18 MiB/s 2024-08-21T01:46:14.3577823Z Receiving objects: 25% (7761/31043), 115.41 MiB | 46.14 MiB/s 2024-08-21T01:46:14.4356593Z Receiving objects: 26% (8072/31043), 115.41 MiB | 46.14 MiB/s 2024-08-21T01:46:14.5136617Z Receiving objects: 27% (8382/31043), 115.41 MiB | 46.14 MiB/s 2024-08-21T01:46:14.5912175Z Receiving objects: 28% (8693/31043), 115.41 MiB | 46.14 MiB/s 2024-08-21T01:46:14.6274531Z Receiving objects: 29% (9003/31043), 115.41 MiB | 46.14 MiB/s 2024-08-21T01:46:14.6697281Z Receiving objects: 29% (9147/31043), 115.41 MiB | 46.14 MiB/s 2024-08-21T01:46:14.7483312Z Receiving objects: 30% (9313/31043), 150.81 MiB | 50.25 MiB/s 2024-08-21T01:46:14.8263176Z Receiving objects: 31% (9624/31043), 150.81 MiB | 50.25 MiB/s 2024-08-21T01:46:15.0858785Z Receiving objects: 32% (9934/31043), 150.81 MiB | 50.25 MiB/s 2024-08-21T01:46:15.1729480Z Receiving objects: 33% (10245/31043), 150.81 MiB | 50.25 MiB/s 2024-08-21T01:46:15.1874306Z Receiving objects: 34% (10555/31043), 182.99 MiB | 52.27 MiB/s 2024-08-21T01:46:15.4818328Z Receiving objects: 35% (10866/31043), 182.99 MiB | 52.27 MiB/s 2024-08-21T01:46:15.6306682Z Receiving objects: 36% (11176/31043), 182.99 MiB | 52.27 MiB/s 2024-08-21T01:46:15.7549472Z Receiving objects: 36% (11266/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7567992Z Receiving objects: 37% (11486/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7594613Z Receiving objects: 38% (11797/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7614475Z Receiving objects: 39% (12107/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7631918Z Receiving objects: 40% (12418/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7643110Z Receiving objects: 41% (12728/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7657353Z Receiving objects: 42% (13039/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7676326Z Receiving objects: 43% (13349/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7692906Z Receiving objects: 44% (13659/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7701915Z Receiving objects: 45% (13970/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7707872Z Receiving objects: 46% (14280/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7724067Z Receiving objects: 47% (14591/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7749482Z Receiving objects: 48% (14901/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.7764041Z Receiving objects: 49% (15212/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8001481Z Receiving objects: 50% (15522/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8006906Z Receiving objects: 51% (15832/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8015161Z Receiving objects: 52% (16143/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8021828Z Receiving objects: 53% (16453/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8027961Z Receiving objects: 54% (16764/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8049224Z Receiving objects: 55% (17074/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8060517Z Receiving objects: 56% (17385/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8067800Z Receiving objects: 57% (17695/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8091196Z Receiving objects: 58% (18005/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8097212Z Receiving objects: 59% (18316/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8109276Z Receiving objects: 60% (18626/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8261017Z Receiving objects: 61% (18937/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8267708Z Receiving objects: 62% (19247/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8274533Z Receiving objects: 63% (19558/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8282694Z Receiving objects: 64% (19868/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8290989Z Receiving objects: 65% (20178/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8300790Z Receiving objects: 66% (20489/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.8585391Z Receiving objects: 67% (20799/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9064466Z Receiving objects: 68% (21110/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9073624Z Receiving objects: 69% (21420/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9088896Z Receiving objects: 70% (21731/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9101285Z Receiving objects: 71% (22041/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9118020Z Receiving objects: 72% (22351/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9127505Z Receiving objects: 73% (22662/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9151927Z Receiving objects: 74% (22972/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9157535Z Receiving objects: 75% (23283/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9166764Z Receiving objects: 76% (23593/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9177776Z Receiving objects: 77% (23904/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9459706Z Receiving objects: 78% (24214/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9501862Z Receiving objects: 79% (24524/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9516533Z Receiving objects: 80% (24835/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9532143Z Receiving objects: 81% (25145/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9550734Z Receiving objects: 82% (25456/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9611194Z Receiving objects: 83% (25766/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9848884Z Receiving objects: 84% (26077/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9856900Z Receiving objects: 85% (26387/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9866477Z Receiving objects: 86% (26697/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9875317Z Receiving objects: 87% (27008/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9885685Z Receiving objects: 88% (27318/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:15.9894124Z Receiving objects: 89% (27629/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0045232Z Receiving objects: 90% (27939/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0063227Z Receiving objects: 91% (28250/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0079425Z Receiving objects: 92% (28560/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0103477Z Receiving objects: 93% (28870/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0151550Z Receiving objects: 94% (29181/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0163028Z Receiving objects: 95% (29491/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0172529Z Receiving objects: 96% (29802/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0181869Z Receiving objects: 97% (30112/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0276617Z Receiving objects: 98% (30423/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0298292Z Receiving objects: 99% (30733/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0299787Z remote: Total 31043 (delta 1038), reused 1496 (delta 835), pack-reused 29260 (from 1) 2024-08-21T01:46:16.0320136Z Receiving objects: 100% (31043/31043), 212.07 MiB | 53.02 MiB/s 2024-08-21T01:46:16.0321114Z Receiving objects: 100% (31043/31043), 232.51 MiB | 52.79 MiB/s, done. 2024-08-21T01:46:16.0362343Z Resolving deltas: 0% (0/16751) 2024-08-21T01:46:16.0378739Z Resolving deltas: 1% (168/16751) 2024-08-21T01:46:16.0395411Z Resolving deltas: 2% (377/16751) 2024-08-21T01:46:16.0427829Z Resolving deltas: 3% (508/16751) 2024-08-21T01:46:16.0446532Z Resolving deltas: 4% (680/16751) 2024-08-21T01:46:16.0456723Z Resolving deltas: 5% (907/16751) 2024-08-21T01:46:16.0468547Z Resolving deltas: 6% (1022/16751) 2024-08-21T01:46:16.0482476Z Resolving deltas: 7% (1173/16751) 2024-08-21T01:46:16.0492185Z Resolving deltas: 8% (1353/16751) 2024-08-21T01:46:16.0506646Z Resolving deltas: 9% (1513/16751) 2024-08-21T01:46:16.0520159Z Resolving deltas: 10% (1677/16751) 2024-08-21T01:46:16.0535441Z Resolving deltas: 11% (1844/16751) 2024-08-21T01:46:16.0550551Z Resolving deltas: 12% (2014/16751) 2024-08-21T01:46:16.0566074Z Resolving deltas: 13% (2199/16751) 2024-08-21T01:46:16.0582881Z Resolving deltas: 14% (2406/16751) 2024-08-21T01:46:16.0603913Z Resolving deltas: 15% (2518/16751) 2024-08-21T01:46:16.0629777Z Resolving deltas: 16% (2697/16751) 2024-08-21T01:46:16.0639429Z Resolving deltas: 18% (3026/16751) 2024-08-21T01:46:16.0653153Z Resolving deltas: 19% (3202/16751) 2024-08-21T01:46:16.0716865Z Resolving deltas: 20% (3365/16751) 2024-08-21T01:46:16.0740594Z Resolving deltas: 21% (3525/16751) 2024-08-21T01:46:16.0741599Z Resolving deltas: 22% (3832/16751) 2024-08-21T01:46:16.0754583Z Resolving deltas: 23% (3858/16751) 2024-08-21T01:46:16.0770015Z Resolving deltas: 24% (4041/16751) 2024-08-21T01:46:16.0784920Z Resolving deltas: 25% (4235/16751) 2024-08-21T01:46:16.0796728Z Resolving deltas: 26% (4430/16751) 2024-08-21T01:46:16.0856221Z Resolving deltas: 27% (4539/16751) 2024-08-21T01:46:16.0870372Z Resolving deltas: 28% (4713/16751) 2024-08-21T01:46:16.0887320Z Resolving deltas: 29% (4862/16751) 2024-08-21T01:46:16.0905324Z Resolving deltas: 30% (5029/16751) 2024-08-21T01:46:16.0924546Z Resolving deltas: 31% (5196/16751) 2024-08-21T01:46:16.0941945Z Resolving deltas: 32% (5365/16751) 2024-08-21T01:46:16.0954038Z Resolving deltas: 33% (5547/16751) 2024-08-21T01:46:16.0967856Z Resolving deltas: 34% (5724/16751) 2024-08-21T01:46:16.0981275Z Resolving deltas: 35% (5900/16751) 2024-08-21T01:46:16.1011681Z Resolving deltas: 36% (6032/16751) 2024-08-21T01:46:16.1031141Z Resolving deltas: 37% (6217/16751) 2024-08-21T01:46:16.1044314Z Resolving deltas: 38% (6377/16751) 2024-08-21T01:46:16.1071315Z Resolving deltas: 39% (6545/16751) 2024-08-21T01:46:16.1087528Z Resolving deltas: 40% (6710/16751) 2024-08-21T01:46:16.1099134Z Resolving deltas: 41% (6875/16751) 2024-08-21T01:46:16.1117974Z Resolving deltas: 42% (7041/16751) 2024-08-21T01:46:16.1131917Z Resolving deltas: 43% (7209/16751) 2024-08-21T01:46:16.1139609Z Resolving deltas: 44% (7378/16751) 2024-08-21T01:46:16.1155507Z Resolving deltas: 45% (7539/16751) 2024-08-21T01:46:16.1176833Z Resolving deltas: 46% (7731/16751) 2024-08-21T01:46:16.1202931Z Resolving deltas: 47% (7875/16751) 2024-08-21T01:46:16.1216911Z Resolving deltas: 48% (8042/16751) 2024-08-21T01:46:16.1247508Z Resolving deltas: 49% (8213/16751) 2024-08-21T01:46:16.1263807Z Resolving deltas: 50% (8405/16751) 2024-08-21T01:46:16.1271029Z Resolving deltas: 51% (8550/16751) 2024-08-21T01:46:16.1281110Z Resolving deltas: 52% (8711/16751) 2024-08-21T01:46:16.1294184Z Resolving deltas: 53% (8891/16751) 2024-08-21T01:46:16.1311950Z Resolving deltas: 54% (9046/16751) 2024-08-21T01:46:16.1327140Z Resolving deltas: 55% (9218/16751) 2024-08-21T01:46:16.1336008Z Resolving deltas: 56% (9382/16751) 2024-08-21T01:46:16.1346777Z Resolving deltas: 57% (9562/16751) 2024-08-21T01:46:16.1360880Z Resolving deltas: 58% (9814/16751) 2024-08-21T01:46:16.1363692Z Resolving deltas: 59% (10005/16751) 2024-08-21T01:46:16.1378234Z Resolving deltas: 60% (10059/16751) 2024-08-21T01:46:16.1379521Z Resolving deltas: 61% (10373/16751) 2024-08-21T01:46:16.1386620Z Resolving deltas: 62% (10413/16751) 2024-08-21T01:46:16.1400347Z Resolving deltas: 63% (10554/16751) 2024-08-21T01:46:16.1424140Z Resolving deltas: 64% (10818/16751) 2024-08-21T01:46:16.1430434Z Resolving deltas: 66% (11127/16751) 2024-08-21T01:46:16.1457256Z Resolving deltas: 67% (11234/16751) 2024-08-21T01:46:16.1464575Z Resolving deltas: 68% (11391/16751) 2024-08-21T01:46:16.1479904Z Resolving deltas: 69% (11559/16751) 2024-08-21T01:46:16.1497888Z Resolving deltas: 70% (11728/16751) 2024-08-21T01:46:16.1511934Z Resolving deltas: 71% (11898/16751) 2024-08-21T01:46:16.1552093Z Resolving deltas: 72% (12072/16751) 2024-08-21T01:46:16.1558572Z Resolving deltas: 75% (12614/16751) 2024-08-21T01:46:16.1575971Z Resolving deltas: 76% (12745/16751) 2024-08-21T01:46:16.1601695Z Resolving deltas: 77% (12899/16751) 2024-08-21T01:46:16.1609922Z Resolving deltas: 78% (13108/16751) 2024-08-21T01:46:16.1624705Z Resolving deltas: 79% (13236/16751) 2024-08-21T01:46:16.1636196Z Resolving deltas: 80% (13450/16751) 2024-08-21T01:46:16.1650219Z Resolving deltas: 81% (13588/16751) 2024-08-21T01:46:16.1662181Z Resolving deltas: 82% (13737/16751) 2024-08-21T01:46:16.1670505Z Resolving deltas: 83% (13919/16751) 2024-08-21T01:46:16.1681283Z Resolving deltas: 84% (14086/16751) 2024-08-21T01:46:16.1689070Z Resolving deltas: 85% (14263/16751) 2024-08-21T01:46:16.1699612Z Resolving deltas: 86% (14430/16751) 2024-08-21T01:46:16.1708259Z Resolving deltas: 87% (14592/16751) 2024-08-21T01:46:16.1719306Z Resolving deltas: 88% (14746/16751) 2024-08-21T01:46:16.1728636Z Resolving deltas: 89% (14914/16751) 2024-08-21T01:46:16.1735852Z Resolving deltas: 90% (15127/16751) 2024-08-21T01:46:16.1746373Z Resolving deltas: 91% (15244/16751) 2024-08-21T01:46:16.1753644Z Resolving deltas: 92% (15428/16751) 2024-08-21T01:46:16.1773202Z Resolving deltas: 93% (15579/16751) 2024-08-21T01:46:16.1774621Z Resolving deltas: 94% (15893/16751) 2024-08-21T01:46:16.1786532Z Resolving deltas: 95% (15918/16751) 2024-08-21T01:46:16.1790973Z Resolving deltas: 96% (16176/16751) 2024-08-21T01:46:16.1800380Z Resolving deltas: 97% (16253/16751) 2024-08-21T01:46:16.1871142Z Resolving deltas: 98% (16416/16751) 2024-08-21T01:46:16.1961033Z Resolving deltas: 99% (16600/16751) 2024-08-21T01:46:16.1962069Z Resolving deltas: 100% (16751/16751) 2024-08-21T01:46:16.1962900Z Resolving deltas: 100% (16751/16751), done. 2024-08-21T01:46:16.8144051Z + pushd torchbench 2024-08-21T01:46:16.8144691Z ~/workspace/torchbench ~/workspace 2024-08-21T01:46:16.8145327Z + git checkout 23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-21T01:46:16.8973921Z Note: switching to '23512dbebd44a11eb84afbf53c3c071dd105297e'. 2024-08-21T01:46:16.8974533Z 2024-08-21T01:46:16.8975087Z You are in 'detached HEAD' state. You can look around, make experimental 2024-08-21T01:46:16.8976173Z changes and commit them, and you can discard any commits you make in this 2024-08-21T01:46:16.8977129Z state without impacting any branches by switching back to a branch. 2024-08-21T01:46:16.8977702Z 2024-08-21T01:46:16.8978106Z If you want to create a new branch to retain commits you create, you may 2024-08-21T01:46:16.8979110Z do so (now or later) by using -c with the switch command. Example: 2024-08-21T01:46:16.8979967Z 2024-08-21T01:46:16.8980262Z git switch -c 2024-08-21T01:46:16.8980985Z 2024-08-21T01:46:16.8981139Z Or undo this operation with: 2024-08-21T01:46:16.8981449Z 2024-08-21T01:46:16.8981573Z git switch - 2024-08-21T01:46:16.8981779Z 2024-08-21T01:46:16.8982210Z Turn off this advice by setting config variable advice.detachedHead to false 2024-08-21T01:46:16.8982832Z 2024-08-21T01:46:16.8983030Z HEAD is now at 23512dbe Pin numpy version 2024-08-21T01:46:16.8983589Z + '[' hf_Bert ']' 2024-08-21T01:46:16.8984448Z + python install.py --continue_on_fail models hf_Bert hf_Albert nanogpt timm_vision_transformer 2024-08-21T01:46:19.8653082Z checking packages torch, torchvision, torchaudio are installed...OK 2024-08-21T01:46:50.8126288Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Albert...OK 2024-08-21T01:46:57.7326786Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert...OK 2024-08-21T01:46:57.7330727Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nanogpt...SKIP - No install.py is found 2024-08-21T01:46:57.7627920Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer...OK 2024-08-21T01:46:58.2983733Z + echo 'Print all dependencies after TorchBench is installed' 2024-08-21T01:46:58.2984591Z Print all dependencies after TorchBench is installed 2024-08-21T01:46:58.2985247Z + python -mpip freeze 2024-08-21T01:46:58.6858157Z absl-py==2.1.0 2024-08-21T01:46:58.6858970Z accelerate==0.33.0 2024-08-21T01:46:58.6859598Z aiohappyeyeballs==2.4.0 2024-08-21T01:46:58.6860375Z aiohttp==3.10.5 2024-08-21T01:46:58.6860954Z aiosignal==1.3.1 2024-08-21T01:46:58.6861971Z astunparse==1.6.3 2024-08-21T01:46:58.6862792Z async-timeout==4.0.3 2024-08-21T01:46:58.6863333Z attrs==24.2.0 2024-08-21T01:46:58.6863651Z audioread==3.0.1 2024-08-21T01:46:58.6864009Z beautifulsoup4==4.12.3 2024-08-21T01:46:58.6868523Z boto3==1.19.12 2024-08-21T01:46:58.6869186Z botocore==1.22.12 2024-08-21T01:46:58.6869620Z bs4==0.0.2 2024-08-21T01:46:58.6870275Z cachetools==5.5.0 2024-08-21T01:46:58.6870876Z certifi==2024.7.4 2024-08-21T01:46:58.6871455Z cffi==1.17.0 2024-08-21T01:46:58.6872386Z charset-normalizer==3.3.2 2024-08-21T01:46:58.6872906Z click==8.1.7 2024-08-21T01:46:58.6873236Z cloudpickle==3.0.0 2024-08-21T01:46:58.6873609Z colorama==0.4.6 2024-08-21T01:46:58.6873941Z coremltools==5.0b5 2024-08-21T01:46:58.6874386Z cryptography==43.0.0 2024-08-21T01:46:58.6874984Z datasets==2.21.0 2024-08-21T01:46:58.6875584Z decorator==5.1.1 2024-08-21T01:46:58.6876369Z Deprecated==1.2.14 2024-08-21T01:46:58.6876946Z dill==0.3.8 2024-08-21T01:46:58.6877467Z distro==1.9.0 2024-08-21T01:46:58.6878055Z exceptiongroup==1.2.2 2024-08-21T01:46:58.6878716Z execnet==2.1.1 2024-08-21T01:46:58.6879407Z expecttest==0.1.6 2024-08-21T01:46:58.6880654Z filelock==3.13.1 2024-08-21T01:46:58.6881260Z flatbuffers==2.0 2024-08-21T01:46:58.6881975Z frozenlist==1.4.1 2024-08-21T01:46:58.6882673Z fsspec==2024.6.1 2024-08-21T01:46:58.6883277Z future==1.0.0 2024-08-21T01:46:58.6883591Z geojson==2.5.0 2024-08-21T01:46:58.6883938Z ghstack==0.8.0 2024-08-21T01:46:58.6884406Z google-auth==2.34.0 2024-08-21T01:46:58.6884844Z google-auth-oauthlib==1.0.0 2024-08-21T01:46:58.6885258Z grpcio==1.65.5 2024-08-21T01:46:58.6885649Z huggingface-hub==0.24.6 2024-08-21T01:46:58.6886107Z hypothesis==5.35.1 2024-08-21T01:46:58.6886440Z idna==3.7 2024-08-21T01:46:58.6886745Z imageio==2.35.1 2024-08-21T01:46:58.6887083Z iniconfig==2.0.0 2024-08-21T01:46:58.6887422Z iopath==0.1.10 2024-08-21T01:46:58.6887735Z Jinja2==3.1.4 2024-08-21T01:46:58.6888060Z jmespath==0.10.0 2024-08-21T01:46:58.6888411Z joblib==1.4.2 2024-08-21T01:46:58.6888734Z junitparser==2.1.1 2024-08-21T01:46:58.6889080Z lark==0.12.0 2024-08-21T01:46:58.6889399Z lazy_loader==0.4 2024-08-21T01:46:58.6889742Z libcst==1.4.0 2024-08-21T01:46:58.6890057Z librosa==0.10.2.post1 2024-08-21T01:46:58.6890460Z lintrunner==0.12.5 2024-08-21T01:46:58.6890805Z llvmlite==0.38.1 2024-08-21T01:46:58.6891125Z lxml==5.0.0 2024-08-21T01:46:58.6891436Z Markdown==3.7 2024-08-21T01:46:58.6892010Z MarkupSafe==2.1.5 2024-08-21T01:46:58.6892413Z ml-dtypes==0.4.0 2024-08-21T01:46:58.6892746Z MonkeyType==23.3.0 2024-08-21T01:46:58.6893092Z mpmath==1.3.0 2024-08-21T01:46:58.6893413Z msgpack==1.0.8 2024-08-21T01:46:58.6893738Z multidict==6.0.5 2024-08-21T01:46:58.6894151Z multiprocess==0.70.16 2024-08-21T01:46:58.6894522Z mypy==1.10.0 2024-08-21T01:46:58.6894905Z mypy-extensions==1.0.0 2024-08-21T01:46:58.6895278Z networkx==2.8.8 2024-08-21T01:46:58.6895607Z ninja==1.10.2 2024-08-21T01:46:58.6895974Z numba==0.55.2 2024-08-21T01:46:58.6896278Z numpy==1.21.2 2024-08-21T01:46:58.6896687Z nvidia-cublas-cu12==12.1.3.1 2024-08-21T01:46:58.6897204Z nvidia-cuda-cupti-cu12==12.1.105 2024-08-21T01:46:58.6897746Z nvidia-cuda-nvrtc-cu12==12.1.105 2024-08-21T01:46:58.6898277Z nvidia-cuda-runtime-cu12==12.1.105 2024-08-21T01:46:58.6898830Z nvidia-cudnn-cu12==9.1.0.70 2024-08-21T01:46:58.6899310Z nvidia-cufft-cu12==11.0.2.54 2024-08-21T01:46:58.6899805Z nvidia-curand-cu12==10.3.2.106 2024-08-21T01:46:58.6900334Z nvidia-cusolver-cu12==11.4.5.107 2024-08-21T01:46:58.6900875Z nvidia-cusparse-cu12==12.1.0.106 2024-08-21T01:46:58.6901380Z nvidia-nccl-cu12==2.20.5 2024-08-21T01:46:58.6901843Z nvidia-nvjitlink-cu12==12.6.20 2024-08-21T01:46:58.6902339Z nvidia-nvtx-cu12==12.1.105 2024-08-21T01:46:58.6902741Z oauthlib==3.2.2 2024-08-21T01:46:58.6903079Z onnx==1.16.1 2024-08-21T01:46:58.6903406Z onnxscript==0.1.0.dev20240817 2024-08-21T01:46:58.6903887Z opencv-python==4.8.0.74 2024-08-21T01:46:58.6904312Z opt-einsum==3.3.0 2024-08-21T01:46:58.6904642Z optree==0.12.1 2024-08-21T01:46:58.6904981Z packaging==24.1 2024-08-21T01:46:58.6905319Z pandas==2.0.3 2024-08-21T01:46:58.6905640Z patch==1.16 2024-08-21T01:46:58.6905978Z pillow==10.3.0 2024-08-21T01:46:58.6906326Z platformdirs==4.2.2 2024-08-21T01:46:58.6906685Z pluggy==1.5.0 2024-08-21T01:46:58.6906993Z pooch==1.8.2 2024-08-21T01:46:58.6907321Z portalocker==2.10.1 2024-08-21T01:46:58.6908051Z protobuf==3.20.2 2024-08-21T01:46:58.6908435Z psutil==6.0.0 2024-08-21T01:46:58.6908839Z py-cpuinfo==9.0.0 2024-08-21T01:46:58.6909191Z pyarrow==17.0.0 2024-08-21T01:46:58.6909594Z pyasn1==0.6.0 2024-08-21T01:46:58.6909929Z pyasn1_modules==0.4.0 2024-08-21T01:46:58.6910293Z pycparser==2.22 2024-08-21T01:46:58.6910639Z PyGithub==2.3.0 2024-08-21T01:46:58.6910978Z Pygments==2.15.0 2024-08-21T01:46:58.6911307Z PyJWT==2.9.0 2024-08-21T01:46:58.6911634Z PyNaCl==1.5.0 2024-08-21T01:46:58.6911956Z pynvml==11.5.3 2024-08-21T01:46:58.6912288Z pytest==7.3.2 2024-08-21T01:46:58.6912665Z pytest-benchmark==4.0.0 2024-08-21T01:46:58.6913103Z pytest-cpp==2.3.0 2024-08-21T01:46:58.6913559Z pytest-flakefinder==1.1.0 2024-08-21T01:46:58.6914027Z pytest-rerunfailures==14.0 2024-08-21T01:46:58.6914488Z pytest-xdist==3.3.1 2024-08-21T01:46:58.6915159Z python-dateutil==2.9.0.post0 2024-08-21T01:46:58.6915583Z pytz==2024.1 2024-08-21T01:46:58.6915947Z PyWavelets==1.4.1 2024-08-21T01:46:58.6916388Z PyYAML @ file:///croot/pyyaml_1698096049011/work 2024-08-21T01:46:58.6916926Z redis==5.0.8 2024-08-21T01:46:58.6917232Z regex==2024.7.24 2024-08-21T01:46:58.6917581Z requests==2.32.3 2024-08-21T01:46:58.6917992Z requests-oauthlib==2.0.0 2024-08-21T01:46:58.6918385Z rockset==1.0.3 2024-08-21T01:46:58.6918694Z rsa==4.9 2024-08-21T01:46:58.6918998Z s3transfer==0.5.2 2024-08-21T01:46:58.6919346Z safetensors==0.4.4 2024-08-21T01:46:58.6919733Z scikit-image==0.22.0 2024-08-21T01:46:58.6920144Z scikit-learn==1.5.1 2024-08-21T01:46:58.6920497Z scipy==1.10.1 2024-08-21T01:46:58.6920835Z sentencepiece==0.2.0 2024-08-21T01:46:58.6921306Z six @ file:///tmp/build/80754af9/six_1644875935023/work 2024-08-21T01:46:58.6921894Z sortedcontainers==2.4.0 2024-08-21T01:46:58.6922279Z soundfile==0.12.1 2024-08-21T01:46:58.6922622Z soupsieve==2.6 2024-08-21T01:46:58.6922940Z soxr==0.4.0 2024-08-21T01:46:58.6923250Z submitit==1.5.1 2024-08-21T01:46:58.6923586Z sympy==1.13.1 2024-08-21T01:46:58.6923896Z tabulate==0.9.0 2024-08-21T01:46:58.6924292Z tb-nightly==2.13.0a20230426 2024-08-21T01:46:58.6924705Z tensorboard==2.13.0 2024-08-21T01:46:58.6925192Z tensorboard-data-server==0.7.2 2024-08-21T01:46:58.6925798Z threadpoolctl==3.5.0 2024-08-21T01:46:58.6926210Z tifffile==2024.8.10 2024-08-21T01:46:58.6927273Z timm @ git+https://github.com/huggingface/pytorch-image-models.git@730b907b4d45a4713cbc425cbf224c46089fd514 2024-08-21T01:46:58.6928259Z tlparse==0.3.25 2024-08-21T01:46:58.6928603Z tokenizers==0.15.2 2024-08-21T01:46:58.6928951Z tomli==2.0.1 2024-08-21T01:46:58.6930522Z torch @ file:///var/lib/jenkins/workspace/dist/torch-2.5.0a0%2Bgit40ec5f6-cp310-cp310-linux_x86_64.whl#sha256=523c3355edb9f5e26124000d42b518512b70158dae8ecb3e1cbd561ad1986734 2024-08-21T01:46:58.6932361Z torchao @ git+https://github.com/pytorch/ao.git@227d4bfece6bb00f0365642ad1d7afd509079ba4 2024-08-21T01:46:58.6933641Z torchaudio @ git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-21T01:46:58.6934938Z torchdata @ git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-21T01:46:58.6936242Z torchtext @ git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-21T01:46:58.6937568Z torchvision @ git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-21T01:46:58.6938557Z tqdm==4.66.5 2024-08-21T01:46:58.6938919Z transformers==4.38.1 2024-08-21T01:46:58.6940009Z -e git+https://github.com/openai/triton@dedb7bdf339a3546896d4820366ca562c586bfa0#egg=triton&subdirectory=python 2024-08-21T01:46:58.6941030Z typing_extensions==4.12.2 2024-08-21T01:46:58.6941417Z tzdata==2024.1 2024-08-21T01:46:58.6941840Z unittest-xml-reporting==3.2.0 2024-08-21T01:46:58.6942274Z urllib3==1.26.19 2024-08-21T01:46:58.6942606Z Werkzeug==3.0.3 2024-08-21T01:46:58.6942978Z wrapt==1.16.0 2024-08-21T01:46:58.6943310Z xdoctest==1.1.0 2024-08-21T01:46:58.6943642Z xxhash==3.5.0 2024-08-21T01:46:58.6943948Z yarl==1.9.4 2024-08-21T01:46:58.6944305Z z3-solver==4.12.2.0 2024-08-21T01:46:58.7279385Z + popd 2024-08-21T01:46:58.7279753Z ~/workspace 2024-08-21T01:46:58.7283724Z ++ pwd 2024-08-21T01:46:58.7287081Z + PYTHONPATH=/var/lib/jenkins/workspace/torchbench 2024-08-21T01:46:58.7287737Z + test_inductor_torchbench_smoketest_perf 2024-08-21T01:46:58.7293996Z ++ pwd 2024-08-21T01:46:58.7298950Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-21T01:46:58.7299922Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-21T01:46:58.7334974Z + TORCHINDUCTOR_ABI_COMPATIBLE=1 2024-08-21T01:46:58.7335512Z + TORCHINDUCTOR_CPP_WRAPPER=1 2024-08-21T01:46:58.7337470Z + python benchmarks/dynamo/torchbench.py --device cuda --accuracy --bfloat16 --inference --inductor --only hf_T5 --output /var/lib/jenkins/workspace/test/test-reports/inductor_cpp_wrapper_inference.csv 2024-08-21T01:47:02.5261209Z 2024-08-21T01:47:03.0729810Z loading model: 0it [00:00, ?it/s] 2024-08-21T01:47:03.0730284Z 2024-08-21T01:47:03.0730847Z config.json: 0% 0.00/1.21k [00:00 will be ignored 2024-08-21T01:49:14.6890600Z [rank0]:W0821 01:49:14.687000 6270 torch/_dynamo/backends/distributed.py:89] [2/0_1] Some buckets were extended beyond their requested parameter capacities in order to ensure each subgraph has an output node, required for fx graph partitioning. This can be the case when a subgraph would have only contained nodes performing inplace mutation, and returning no logical outputs. This should not be a problem, unless it results in too few graph partitions for optimal DDP performance. 2024-08-21T01:49:14.7016250Z [rank0]:W0821 01:49:14.700000 6270 torch/_dynamo/backends/distributed.py:106] [2/0_1] DDPOptimizer extended these buckets to ensure per-subgraph output nodes: 2024-08-21T01:49:14.7026297Z [rank0]:W0821 01:49:14.700000 6270 torch/_dynamo/backends/distributed.py:106] [2/0_1] ┌─────────┬─────────────┬────────────────────────┐ 2024-08-21T01:49:14.7028934Z [rank0]:W0821 01:49:14.700000 6270 torch/_dynamo/backends/distributed.py:106] [2/0_1] │ Index │ Extra Ops │ Extra Param Size (b) │ 2024-08-21T01:49:14.7030979Z [rank0]:W0821 01:49:14.700000 6270 torch/_dynamo/backends/distributed.py:106] [2/0_1] ├─────────┼─────────────┼────────────────────────┤ 2024-08-21T01:49:14.7033104Z [rank0]:W0821 01:49:14.700000 6270 torch/_dynamo/backends/distributed.py:106] [2/0_1] │ 0 │ 470 │ 44910720 │ 2024-08-21T01:49:14.7034966Z [rank0]:W0821 01:49:14.700000 6270 torch/_dynamo/backends/distributed.py:106] [2/0_1] └─────────┴─────────────┴────────────────────────┘ 2024-08-21T01:50:54.5992593Z [rank0]:W0821 01:50:54.597000 6270 torch/_inductor/utils.py:1391] [3/0_1] DeviceCopy in input program 2024-08-21T01:51:09.0896892Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] Graph break from `Tensor.item()`, consider setting: 2024-08-21T01:51:09.0899371Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] torch._dynamo.config.capture_scalar_outputs = True 2024-08-21T01:51:09.0901482Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] or: 2024-08-21T01:51:09.0903515Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-08-21T01:51:09.0906969Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] to include these operations in the captured graph. 2024-08-21T01:51:09.0909434Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] 2024-08-21T01:51:09.0911321Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] Graph break: from user code at: 2024-08-21T01:51:09.0914905Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco/moco/builder.py", line 158, in torch_dynamo_resume_in_forward_at_133 2024-08-21T01:51:09.0918516Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] self._dequeue_and_enqueue(k) 2024-08-21T01:51:09.0923263Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context 2024-08-21T01:51:09.0927227Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] return func(*args, **kwargs) 2024-08-21T01:51:09.0931442Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco/moco/builder.py", line 59, in _dequeue_and_enqueue 2024-08-21T01:51:09.0935973Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] ptr = int(self.queue_ptr) 2024-08-21T01:51:09.0937939Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] 2024-08-21T01:51:09.0939901Z [rank0]:W0821 01:51:09.088000 6270 torch/_dynamo/variables/tensor.py:776] [5/0] 2024-08-21T01:51:56.5196471Z pass 2024-08-21T01:51:57.6519971Z [rank0]:[W821 01:51:57.344295137 ProcessGroupNCCL.cpp:1223] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator()) 2024-08-21T01:51:59.5552105Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_cpp_wrapper_inference.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_torchbench_inference.csv 2024-08-21T01:52:00.0688722Z hf_T5 PASS 2024-08-21T01:52:00.0693605Z llama PASS 2024-08-21T01:52:00.0699428Z moco PASS 2024-08-21T01:52:00.1370922Z ++ realpath benchmarks/dynamo/torchbench_models_list.txt 2024-08-21T01:52:00.1396873Z + python benchmarks/dynamo/torchbench.py --device cuda --performance --backend inductor --float16 --training --batch-size-file /var/lib/jenkins/workspace/benchmarks/dynamo/torchbench_models_list.txt --only hf_Bert --output /var/lib/jenkins/workspace/test/test-reports/inductor_training_smoketest.csv 2024-08-21T01:52:03.9323980Z batch size: 16 2024-08-21T01:52:03.9334261Z 2024-08-21T01:52:07.9812927Z loading model: 0it [00:00, ?it/s] 2024-08-21T01:52:07.9813549Z loading model: 0it [00:04, ?it/s] 2024-08-21T01:52:07.9860873Z cuda train hf_Bert 2024-08-21T01:52:49.5724872Z W0821 01:52:49.570000 6853 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T01:53:42.2179350Z 2024-08-21T01:53:42.4569352Z running benchmark: 0% 0/30 [00:00 will be ignored 2024-08-21T01:55:36.5041969Z 2024-08-21T01:55:36.9319561Z running benchmark: 0% 0/30 [00:00 will be ignored 2024-08-21T01:57:50.8790664Z pass 2024-08-21T01:57:53.5219212Z Performing warm-start run for AlbertForQuestionAnswering 2024-08-21T01:57:57.7068709Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T01:57:57.7071270Z warnings.warn( 2024-08-21T01:57:58.3193348Z 2024-08-21T01:58:02.4317839Z loading model: 0it [00:00, ?it/s] 2024-08-21T01:58:02.4318809Z loading model: 0it [00:04, ?it/s] 2024-08-21T01:58:02.4319648Z cuda train AlbertForQuestionAnswering 2024-08-21T01:58:21.6129174Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T01:58:21.6130745Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T01:58:21.6132059Z pred = mod(**cloned_inputs) 2024-08-21T01:58:21.6133599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/albert/modeling_albert.py", line 1256, in forward 2024-08-21T01:58:21.6134776Z outputs = self.albert( 2024-08-21T01:58:21.6136051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/albert/modeling_albert.py", line 729, in forward 2024-08-21T01:58:21.6138719Z embedding_output = self.embeddings( 2024-08-21T01:58:21.6140323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/albert/modeling_albert.py", line 250, in forward 2024-08-21T01:58:21.6141653Z inputs_embeds = self.word_embeddings(input_ids) 2024-08-21T01:58:21.6142088Z 2024-08-21T01:58:22.3363729Z W0821 01:58:22.335000 9343 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T01:58:25.9015385Z pass 2024-08-21T01:58:28.1204897Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_AlbertForQuestionAnswering.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_huggingface_training.csv 2024-08-21T01:58:28.6049322Z AlbertForQuestionAnswering PASS 2024-08-21T01:58:28.6703306Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-08-21T01:58:28.6706547Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only AllenaiLongformerBase --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_AllenaiLongformerBase.csv 2024-08-21T01:58:32.8305344Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T01:58:32.8307771Z warnings.warn( 2024-08-21T01:58:33.2915426Z Performing cold-start run for AllenaiLongformerBase 2024-08-21T01:58:37.4151536Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T01:58:37.4154553Z warnings.warn( 2024-08-21T01:58:38.0443159Z 2024-08-21T01:58:42.0999107Z loading model: 0it [00:00, ?it/s] 2024-08-21T01:58:42.0999682Z loading model: 0it [00:04, ?it/s] 2024-08-21T01:58:42.1000229Z cuda train AllenaiLongformerBase 2024-08-21T01:58:45.3687156Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] Graph break from `Tensor.item()`, consider setting: 2024-08-21T01:58:45.3689366Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] torch._dynamo.config.capture_scalar_outputs = True 2024-08-21T01:58:45.3692016Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] or: 2024-08-21T01:58:45.3693365Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-08-21T01:58:45.3694918Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] to include these operations in the captured graph. 2024-08-21T01:58:45.3696179Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] 2024-08-21T01:58:45.3697289Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] Graph break: from user code at: 2024-08-21T01:58:45.3699481Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T01:58:45.3701612Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] pred = mod(**cloned_inputs) 2024-08-21T01:58:45.3704176Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1835, in forward 2024-08-21T01:58:45.3706274Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] outputs = self.longformer( 2024-08-21T01:58:45.3708819Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1738, in forward 2024-08-21T01:58:45.3710943Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] encoder_outputs = self.encoder( 2024-08-21T01:58:45.3713317Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1291, in forward 2024-08-21T01:58:45.3716013Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] is_global_attn = is_index_global_attn.flatten().any().item() 2024-08-21T01:58:45.3717324Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] 2024-08-21T01:58:45.3718208Z W0821 01:58:45.367000 9482 torch/_dynamo/variables/tensor.py:776] [3/0] 2024-08-21T02:00:26.0445299Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:00:26.0450567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1318, in torch_dynamo_resume_in_forward_at_1291 2024-08-21T02:00:26.0452080Z layer_outputs = layer_module( 2024-08-21T02:00:26.0453680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1246, in forward 2024-08-21T02:00:26.0455049Z self_attn_outputs = self.attention( 2024-08-21T02:00:26.0456502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1182, in forward 2024-08-21T02:00:26.0457723Z self_outputs = self.self( 2024-08-21T02:00:26.0459116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 571, in forward 2024-08-21T02:00:26.0460869Z attn_scores = self._sliding_chunks_query_key_matmul( 2024-08-21T02:00:26.0462562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 836, in _sliding_chunks_query_key_matmul 2024-08-21T02:00:26.0464270Z query = self._chunk(query, window_overlap, getattr(self.config, "onnx_export", False)) 2024-08-21T02:00:26.0466004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 778, in _chunk 2024-08-21T02:00:26.0467665Z return hidden_states.as_strided(size=chunk_size, stride=chunk_stride) 2024-08-21T02:00:26.0468271Z 2024-08-21T02:00:26.6835691Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:00:26.6837770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1734, in forward 2024-08-21T02:00:26.6839143Z embedding_output = self.embeddings( 2024-08-21T02:00:26.6840588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 470, in forward 2024-08-21T02:00:26.6841885Z inputs_embeds = self.word_embeddings(input_ids) 2024-08-21T02:00:26.6842332Z 2024-08-21T02:00:26.6940011Z W0821 02:00:26.693000 9482 torch/_logging/_internal.py:1058] [16/0] Profiler function will be ignored 2024-08-21T02:01:23.6354498Z pass 2024-08-21T02:01:29.7151213Z Performing warm-start run for AllenaiLongformerBase 2024-08-21T02:01:33.8713530Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:01:33.8715812Z warnings.warn( 2024-08-21T02:01:34.4653863Z 2024-08-21T02:01:38.4980348Z loading model: 0it [00:00, ?it/s] 2024-08-21T02:01:38.4980962Z loading model: 0it [00:04, ?it/s] 2024-08-21T02:01:38.4981510Z cuda train AllenaiLongformerBase 2024-08-21T02:01:41.7332440Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] Graph break from `Tensor.item()`, consider setting: 2024-08-21T02:01:41.7338649Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] torch._dynamo.config.capture_scalar_outputs = True 2024-08-21T02:01:41.7340963Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] or: 2024-08-21T02:01:41.7342661Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1 2024-08-21T02:01:41.7344653Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] to include these operations in the captured graph. 2024-08-21T02:01:41.7345997Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] 2024-08-21T02:01:41.7347145Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] Graph break: from user code at: 2024-08-21T02:01:41.7349530Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T02:01:41.7351885Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] pred = mod(**cloned_inputs) 2024-08-21T02:01:41.7354456Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1835, in forward 2024-08-21T02:01:41.7356639Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] outputs = self.longformer( 2024-08-21T02:01:41.7358981Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1738, in forward 2024-08-21T02:01:41.7361367Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] encoder_outputs = self.encoder( 2024-08-21T02:01:41.7363824Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1291, in forward 2024-08-21T02:01:41.7366142Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] is_global_attn = is_index_global_attn.flatten().any().item() 2024-08-21T02:01:41.7367485Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] 2024-08-21T02:01:41.7368408Z W0821 02:01:41.732000 10226 torch/_dynamo/variables/tensor.py:776] [3/0] 2024-08-21T02:02:23.0812189Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:02:23.0815906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1318, in torch_dynamo_resume_in_forward_at_1291 2024-08-21T02:02:23.0818241Z layer_outputs = layer_module( 2024-08-21T02:02:23.0820676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1246, in forward 2024-08-21T02:02:23.0823096Z self_attn_outputs = self.attention( 2024-08-21T02:02:23.0825469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1182, in forward 2024-08-21T02:02:23.0827802Z self_outputs = self.self( 2024-08-21T02:02:23.0830408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 571, in forward 2024-08-21T02:02:23.0832670Z attn_scores = self._sliding_chunks_query_key_matmul( 2024-08-21T02:02:23.0835503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 836, in _sliding_chunks_query_key_matmul 2024-08-21T02:02:23.0838616Z query = self._chunk(query, window_overlap, getattr(self.config, "onnx_export", False)) 2024-08-21T02:02:23.0841725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 778, in _chunk 2024-08-21T02:02:23.0844299Z return hidden_states.as_strided(size=chunk_size, stride=chunk_stride) 2024-08-21T02:02:23.0845333Z 2024-08-21T02:02:23.3152532Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:02:23.3155409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1734, in forward 2024-08-21T02:02:23.3157958Z embedding_output = self.embeddings( 2024-08-21T02:02:23.3160477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 470, in forward 2024-08-21T02:02:23.3162389Z inputs_embeds = self.word_embeddings(input_ids) 2024-08-21T02:02:23.3162996Z 2024-08-21T02:02:23.3251438Z W0821 02:02:23.324000 10226 torch/_logging/_internal.py:1058] [16/0] Profiler function will be ignored 2024-08-21T02:02:44.6203800Z pass 2024-08-21T02:02:47.1010013Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_AllenaiLongformerBase.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_huggingface_training.csv 2024-08-21T02:02:47.5819981Z AllenaiLongformerBase PASS 2024-08-21T02:02:47.6446465Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-08-21T02:02:47.6449727Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only DistilBertForMaskedLM --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_DistilBertForMaskedLM.csv 2024-08-21T02:02:51.7772110Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:02:51.7774291Z warnings.warn( 2024-08-21T02:02:52.2783799Z Performing cold-start run for DistilBertForMaskedLM 2024-08-21T02:02:56.4422185Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:02:56.4424532Z warnings.warn( 2024-08-21T02:02:57.1489330Z 2024-08-21T02:02:59.3335661Z loading model: 0it [00:00, ?it/s] 2024-08-21T02:02:59.3336509Z loading model: 0it [00:02, ?it/s] 2024-08-21T02:02:59.3337419Z cuda train DistilBertForMaskedLM 2024-08-21T02:03:26.9001259Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:03:26.9002806Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T02:03:26.9004106Z pred = mod(**cloned_inputs) 2024-08-21T02:03:26.9005754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 905, in forward 2024-08-21T02:03:26.9007002Z dlbrt_output = self.distilbert( 2024-08-21T02:03:26.9008484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2024-08-21T02:03:26.9010026Z embeddings = self.embeddings(input_ids, inputs_embeds) # (bs, seq_length, dim) 2024-08-21T02:03:26.9011703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 141, in forward 2024-08-21T02:03:26.9013166Z input_embeds = self.word_embeddings(input_ids) # (bs, max_seq_length, dim) 2024-08-21T02:03:26.9013805Z 2024-08-21T02:03:27.0206103Z W0821 02:03:27.019000 10365 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T02:03:55.4872825Z pass 2024-08-21T02:03:58.2750442Z Performing warm-start run for DistilBertForMaskedLM 2024-08-21T02:04:02.4325413Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:04:02.4329184Z warnings.warn( 2024-08-21T02:04:03.2297607Z 2024-08-21T02:04:05.4073499Z loading model: 0it [00:00, ?it/s] 2024-08-21T02:04:05.4074211Z loading model: 0it [00:02, ?it/s] 2024-08-21T02:04:05.4074758Z cuda train DistilBertForMaskedLM 2024-08-21T02:04:17.8861740Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:04:17.8863471Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T02:04:17.8864767Z pred = mod(**cloned_inputs) 2024-08-21T02:04:17.8867242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 905, in forward 2024-08-21T02:04:17.8869809Z dlbrt_output = self.distilbert( 2024-08-21T02:04:17.8871644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2024-08-21T02:04:17.8873282Z embeddings = self.embeddings(input_ids, inputs_embeds) # (bs, seq_length, dim) 2024-08-21T02:04:17.8874962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 141, in forward 2024-08-21T02:04:17.8876505Z input_embeds = self.word_embeddings(input_ids) # (bs, max_seq_length, dim) 2024-08-21T02:04:17.8877556Z 2024-08-21T02:04:18.0069513Z W0821 02:04:18.005000 10914 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T02:04:27.8767873Z pass 2024-08-21T02:04:30.1014047Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_DistilBertForMaskedLM.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_huggingface_training.csv 2024-08-21T02:04:30.5833808Z DistilBertForMaskedLM PASS 2024-08-21T02:04:30.6473149Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-08-21T02:04:30.6476279Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only DistillGPT2 --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_DistillGPT2.csv 2024-08-21T02:04:35.2234780Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:04:35.2237271Z warnings.warn( 2024-08-21T02:04:35.6439200Z Performing cold-start run for DistillGPT2 2024-08-21T02:04:39.7814373Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:04:39.7816705Z warnings.warn( 2024-08-21T02:04:40.3643256Z 2024-08-21T02:04:43.3095719Z loading model: 0it [00:00, ?it/s] 2024-08-21T02:04:43.3096637Z loading model: 0it [00:02, ?it/s] 2024-08-21T02:04:43.3097417Z cuda train DistillGPT2 2024-08-21T02:05:09.3629888Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:05:09.3633821Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T02:05:09.3636069Z pred = mod(**cloned_inputs) 2024-08-21T02:05:09.3638156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1074, in forward 2024-08-21T02:05:09.3639457Z transformer_outputs = self.transformer( 2024-08-21T02:05:09.3640803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 837, in forward 2024-08-21T02:05:09.3642371Z inputs_embeds = self.wte(input_ids) 2024-08-21T02:05:09.3642762Z 2024-08-21T02:05:09.4660889Z W0821 02:05:09.464000 11053 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T02:05:29.6994015Z pass 2024-08-21T02:05:32.3500757Z Performing warm-start run for DistillGPT2 2024-08-21T02:05:37.0385122Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:05:37.0388360Z warnings.warn( 2024-08-21T02:05:37.6583801Z 2024-08-21T02:05:40.5982823Z loading model: 0it [00:00, ?it/s] 2024-08-21T02:05:40.5983418Z loading model: 0it [00:02, ?it/s] 2024-08-21T02:05:40.5984009Z cuda train DistillGPT2 2024-08-21T02:05:52.5253909Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:05:52.5255893Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T02:05:52.5257862Z pred = mod(**cloned_inputs) 2024-08-21T02:05:52.5260393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1074, in forward 2024-08-21T02:05:52.5263149Z transformer_outputs = self.transformer( 2024-08-21T02:05:52.5265439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 837, in forward 2024-08-21T02:05:52.5267742Z inputs_embeds = self.wte(input_ids) 2024-08-21T02:05:52.5268438Z 2024-08-21T02:05:52.6267086Z W0821 02:05:52.625000 11552 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T02:06:00.1355308Z pass 2024-08-21T02:06:02.3773427Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_DistillGPT2.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_huggingface_training.csv 2024-08-21T02:06:02.8556427Z DistillGPT2 PASS 2024-08-21T02:06:02.9189983Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-08-21T02:06:02.9193000Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only GoogleFnet --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_GoogleFnet.csv 2024-08-21T02:06:07.0409997Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:06:07.0412287Z warnings.warn( 2024-08-21T02:06:07.6494691Z Performing cold-start run for GoogleFnet 2024-08-21T02:06:11.7653751Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:06:11.7656072Z warnings.warn( 2024-08-21T02:06:12.5151565Z 2024-08-21T02:06:14.6321124Z loading model: 0it [00:00, ?it/s]WARNING:common:Model GoogleFnet supports float32 only 2024-08-21T02:06:15.0377565Z 2024-08-21T02:06:15.0378538Z loading model: 0it [00:02, ?it/s] 2024-08-21T02:06:15.0379230Z WARNING:common:Model GoogleFnet supports float32 only 2024-08-21T02:06:15.0390565Z cuda train GoogleFnet 2024-08-21T02:06:15.3233286Z WARNING:common:Model GoogleFnet supports float32 only 2024-08-21T02:06:42.4604631Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:06:42.4606931Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T02:06:42.4608248Z pred = mod(**cloned_inputs) 2024-08-21T02:06:42.4610435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 749, in forward 2024-08-21T02:06:42.4611648Z outputs = self.fnet( 2024-08-21T02:06:42.4612835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 581, in forward 2024-08-21T02:06:42.4614016Z embedding_output = self.embeddings( 2024-08-21T02:06:42.4615312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 148, in forward 2024-08-21T02:06:42.4616583Z inputs_embeds = self.word_embeddings(input_ids) 2024-08-21T02:06:42.4617016Z 2024-08-21T02:06:42.5866668Z W0821 02:06:42.585000 11691 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T02:07:10.9508237Z pass 2024-08-21T02:07:13.9263106Z Performing warm-start run for GoogleFnet 2024-08-21T02:07:18.0673987Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:07:18.0676992Z warnings.warn( 2024-08-21T02:07:18.6879869Z 2024-08-21T02:07:20.8090881Z loading model: 0it [00:00, ?it/s]WARNING:common:Model GoogleFnet supports float32 only 2024-08-21T02:07:21.0833235Z 2024-08-21T02:07:21.0833814Z loading model: 0it [00:02, ?it/s] 2024-08-21T02:07:21.0834492Z WARNING:common:Model GoogleFnet supports float32 only 2024-08-21T02:07:21.0848014Z cuda train GoogleFnet 2024-08-21T02:07:21.2652720Z WARNING:common:Model GoogleFnet supports float32 only 2024-08-21T02:07:32.9683041Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:07:32.9685741Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T02:07:32.9687895Z pred = mod(**cloned_inputs) 2024-08-21T02:07:32.9689518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 749, in forward 2024-08-21T02:07:32.9690656Z outputs = self.fnet( 2024-08-21T02:07:32.9691844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 581, in forward 2024-08-21T02:07:32.9693002Z embedding_output = self.embeddings( 2024-08-21T02:07:32.9694367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 148, in forward 2024-08-21T02:07:32.9695580Z inputs_embeds = self.word_embeddings(input_ids) 2024-08-21T02:07:32.9696038Z 2024-08-21T02:07:33.0861110Z W0821 02:07:33.085000 12174 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T02:07:43.4839211Z pass 2024-08-21T02:07:45.6832745Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_GoogleFnet.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_huggingface_training.csv 2024-08-21T02:07:46.1651497Z GoogleFnet PASS 2024-08-21T02:07:46.2279111Z + for test in AlbertForQuestionAnswering AllenaiLongformerBase DistilBertForMaskedLM DistillGPT2 GoogleFnet YituTechConvBert 2024-08-21T02:07:46.2284423Z + python benchmarks/dynamo/huggingface.py --accuracy --training --amp --inductor --device cuda --warm-start-latency --only YituTechConvBert --output /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_YituTechConvBert.csv 2024-08-21T02:07:50.3518572Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:07:50.3520783Z warnings.warn( 2024-08-21T02:07:50.7273458Z Performing cold-start run for YituTechConvBert 2024-08-21T02:07:54.8339096Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:07:54.8341327Z warnings.warn( 2024-08-21T02:07:55.4251348Z 2024-08-21T02:07:58.5899684Z loading model: 0it [00:00, ?it/s] 2024-08-21T02:07:58.5900414Z loading model: 0it [00:03, ?it/s] 2024-08-21T02:07:58.5901002Z cuda train YituTechConvBert 2024-08-21T02:09:03.9091078Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:09:03.9093105Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T02:09:03.9094599Z pred = mod(**cloned_inputs) 2024-08-21T02:09:03.9096931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 918, in forward 2024-08-21T02:09:03.9098634Z generator_hidden_states = self.convbert( 2024-08-21T02:09:03.9100058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 834, in forward 2024-08-21T02:09:03.9101272Z hidden_states = self.embeddings( 2024-08-21T02:09:03.9102621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 230, in forward 2024-08-21T02:09:03.9103874Z inputs_embeds = self.word_embeddings(input_ids) 2024-08-21T02:09:03.9104317Z 2024-08-21T02:09:04.0656403Z W0821 02:09:04.064000 12312 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T02:10:18.2420822Z pass 2024-08-21T02:10:23.9883767Z Performing warm-start run for YituTechConvBert 2024-08-21T02:10:28.1252431Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`. 2024-08-21T02:10:28.1256329Z warnings.warn( 2024-08-21T02:10:28.8568329Z 2024-08-21T02:10:31.9936135Z loading model: 0it [00:00, ?it/s] 2024-08-21T02:10:31.9936967Z loading model: 0it [00:03, ?it/s] 2024-08-21T02:10:31.9937877Z cuda train YituTechConvBert 2024-08-21T02:11:01.4111495Z skipping cudagraphs due to deterministic index put. Found from : 2024-08-21T02:11:01.4113100Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 524, in torch_dynamo_resume_in_forward_and_backward_pass_at_522 2024-08-21T02:11:01.4115139Z pred = mod(**cloned_inputs) 2024-08-21T02:11:01.4118103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 918, in forward 2024-08-21T02:11:01.4119444Z generator_hidden_states = self.convbert( 2024-08-21T02:11:01.4120860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 834, in forward 2024-08-21T02:11:01.4122071Z hidden_states = self.embeddings( 2024-08-21T02:11:01.4123407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 230, in forward 2024-08-21T02:11:01.4124684Z inputs_embeds = self.word_embeddings(input_ids) 2024-08-21T02:11:01.4125129Z 2024-08-21T02:11:01.5623836Z W0821 02:11:01.561000 13026 torch/_logging/_internal.py:1058] [4/0] Profiler function will be ignored 2024-08-21T02:11:27.5646681Z pass 2024-08-21T02:11:30.0131337Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inductor_warm_start_smoketest_YituTechConvBert.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_huggingface_training.csv 2024-08-21T02:11:30.4873975Z YituTechConvBert PASS 2024-08-21T02:11:30.5503706Z + cleanup_workspace 2024-08-21T02:11:30.5505440Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-21T02:11:30.5508121Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-21T02:11:30.5509490Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-21T02:11:30.5510273Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-21T02:11:30.5511464Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-21T02:11:30.5512633Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-21T02:11:30.5513504Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-21T02:11:31.2121320Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-21T02:11:31.2121947Z cat test/**/*_toprint.log || true 2024-08-21T02:11:31.2135203Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T02:11:31.2135866Z env: 2024-08-21T02:11:31.2136169Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:31.2136711Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:31.2137631Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:31.2138546Z ##[endgroup] 2024-08-21T02:11:31.2254910Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-21T02:11:31.2292072Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-21T02:11:31.2292700Z kill "$MONITOR_SCRIPT_PID" 2024-08-21T02:11:31.2305262Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T02:11:31.2305884Z env: 2024-08-21T02:11:31.2306205Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:31.2306751Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:31.2308046Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:31.2308955Z MONITOR_SCRIPT_PID: 1151399 2024-08-21T02:11:31.2309383Z ##[endgroup] 2024-08-21T02:11:31.2575389Z Prepare all required actions 2024-08-21T02:11:31.2576066Z Getting action download info 2024-08-21T02:11:31.5814922Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-21T02:11:32.0883034Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-21T02:11:32.0883654Z with: 2024-08-21T02:11:32.0884302Z file-suffix: test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954 2024-08-21T02:11:32.0885176Z use-gha: anything-non-empty-to-use-gha 2024-08-21T02:11:32.0885707Z s3-bucket: gha-artifacts 2024-08-21T02:11:32.0886089Z env: 2024-08-21T02:11:32.0886403Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:32.0886947Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:32.0887871Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:32.0888724Z ##[endgroup] 2024-08-21T02:11:32.0987039Z ##[group]Run actions/upload-artifact@v3 2024-08-21T02:11:32.0987886Z with: 2024-08-21T02:11:32.0988927Z name: test-jsons-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954.zip 2024-08-21T02:11:32.0989923Z retention-days: 14 2024-08-21T02:11:32.0990318Z if-no-files-found: warn 2024-08-21T02:11:32.0990734Z path: test/**/*.json 2024-08-21T02:11:32.0991089Z env: 2024-08-21T02:11:32.0991395Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:32.0991936Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:32.0992865Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:32.0993656Z ##[endgroup] 2024-08-21T02:11:32.5697120Z With the provided path, there will be 5 files uploaded 2024-08-21T02:11:32.5700140Z Starting artifact upload 2024-08-21T02:11:32.5703245Z For more detailed logs during the artifact upload process, enable step-debugging: https://docs.github.com/actions/monitoring-and-troubleshooting-workflows/enabling-debug-logging#enabling-step-debug-logging 2024-08-21T02:11:32.5705084Z Artifact name is valid! 2024-08-21T02:11:32.8029084Z Container for artifact "test-jsons-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954.zip" successfully created. Starting upload of file(s) 2024-08-21T02:11:33.1548310Z Total size of all the files uploaded is 40773 bytes 2024-08-21T02:11:33.1549684Z File upload process has finished. Finalizing the artifact upload 2024-08-21T02:11:33.2010098Z Artifact has been finalized. All files have been successfully uploaded! 2024-08-21T02:11:33.2010746Z 2024-08-21T02:11:33.2011172Z The raw size of all the files that were specified for upload is 361240 bytes 2024-08-21T02:11:33.2012787Z The size of all the files that were uploaded is 40773 bytes. This takes into account any gzip compression used to reduce the upload size, time and storage 2024-08-21T02:11:33.2014261Z 2024-08-21T02:11:33.2016388Z Note: The size of downloaded zips can differ significantly from the reported size. For more information see: https://github.com/actions/upload-artifact#zipped-artifact-downloads 2024-08-21T02:11:33.2017748Z 2024-08-21T02:11:33.2018881Z Artifact test-jsons-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954.zip has been successfully uploaded! 2024-08-21T02:11:33.2141537Z ##[group]Run actions/upload-artifact@v3 2024-08-21T02:11:33.2142049Z with: 2024-08-21T02:11:33.2142897Z name: test-reports-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954.zip 2024-08-21T02:11:33.2143914Z retention-days: 14 2024-08-21T02:11:33.2144300Z if-no-files-found: ignore 2024-08-21T02:11:33.2144748Z path: test/**/*.xml test/**/*.csv 2024-08-21T02:11:33.2145205Z env: 2024-08-21T02:11:33.2145516Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:33.2146099Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:33.2147022Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:33.2148049Z ##[endgroup] 2024-08-21T02:11:33.6897159Z With the provided path, there will be 15 files uploaded 2024-08-21T02:11:33.6899142Z Starting artifact upload 2024-08-21T02:11:33.6901070Z For more detailed logs during the artifact upload process, enable step-debugging: https://docs.github.com/actions/monitoring-and-troubleshooting-workflows/enabling-debug-logging#enabling-step-debug-logging 2024-08-21T02:11:33.6902841Z Artifact name is valid! 2024-08-21T02:11:33.9119250Z Container for artifact "test-reports-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954.zip" successfully created. Starting upload of file(s) 2024-08-21T02:11:34.7193414Z Total size of all the files uploaded is 3175 bytes 2024-08-21T02:11:34.7194269Z File upload process has finished. Finalizing the artifact upload 2024-08-21T02:11:34.7630254Z Artifact has been finalized. All files have been successfully uploaded! 2024-08-21T02:11:34.7631059Z 2024-08-21T02:11:34.7631454Z The raw size of all the files that were specified for upload is 4531 bytes 2024-08-21T02:11:34.7633037Z The size of all the files that were uploaded is 3175 bytes. This takes into account any gzip compression used to reduce the upload size, time and storage 2024-08-21T02:11:34.7634190Z 2024-08-21T02:11:34.7635543Z Note: The size of downloaded zips can differ significantly from the reported size. For more information see: https://github.com/actions/upload-artifact#zipped-artifact-downloads 2024-08-21T02:11:34.7636941Z 2024-08-21T02:11:34.7638047Z Artifact test-reports-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954.zip has been successfully uploaded! 2024-08-21T02:11:34.7754285Z ##[group]Run actions/upload-artifact@v3 2024-08-21T02:11:34.7754794Z with: 2024-08-21T02:11:34.7755556Z name: logs-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954.zip 2024-08-21T02:11:34.7756501Z retention-days: 14 2024-08-21T02:11:34.7756904Z if-no-files-found: ignore 2024-08-21T02:11:34.7757382Z path: usage_log.txt test/**/*.log 2024-08-21T02:11:34.7757910Z env: 2024-08-21T02:11:34.7758262Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:34.7758791Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:34.7759708Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:34.7760507Z ##[endgroup] 2024-08-21T02:11:35.3136908Z Multiple search paths detected. Calculating the least common ancestor of all paths 2024-08-21T02:11:35.3140301Z The least common ancestor is /home/weiwangmeta/actions-runner/_work/pytorch/pytorch. This will be the root directory of the artifact 2024-08-21T02:11:35.3141632Z With the provided path, there will be 1 file uploaded 2024-08-21T02:11:35.3142800Z Starting artifact upload 2024-08-21T02:11:35.3144784Z For more detailed logs during the artifact upload process, enable step-debugging: https://docs.github.com/actions/monitoring-and-troubleshooting-workflows/enabling-debug-logging#enabling-step-debug-logging 2024-08-21T02:11:35.3146970Z Artifact name is valid! 2024-08-21T02:11:35.5319484Z Container for artifact "logs-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954.zip" successfully created. Starting upload of file(s) 2024-08-21T02:11:35.6810911Z Total size of all the files uploaded is 15225 bytes 2024-08-21T02:11:35.6811758Z File upload process has finished. Finalizing the artifact upload 2024-08-21T02:11:35.7267147Z Artifact has been finalized. All files have been successfully uploaded! 2024-08-21T02:11:35.7267979Z 2024-08-21T02:11:35.7268460Z The raw size of all the files that were specified for upload is 192192 bytes 2024-08-21T02:11:35.7270035Z The size of all the files that were uploaded is 15225 bytes. This takes into account any gzip compression used to reduce the upload size, time and storage 2024-08-21T02:11:35.7271179Z 2024-08-21T02:11:35.7272974Z Note: The size of downloaded zips can differ significantly from the reported size. For more information see: https://github.com/actions/upload-artifact#zipped-artifact-downloads 2024-08-21T02:11:35.7274332Z 2024-08-21T02:11:35.7275352Z Artifact logs-runattempt1-test-inductor_torchbench_smoketest_perf-1-1-linux.gcp.a100_29026270954.zip has been successfully uploaded! 2024-08-21T02:11:35.7380813Z ##[group]Run # shellcheck disable=SC2156 2024-08-21T02:11:35.7381394Z # shellcheck disable=SC2156 2024-08-21T02:11:35.7382403Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-21T02:11:35.7396402Z shell: /usr/bin/bash -e {0} 2024-08-21T02:11:35.7396824Z env: 2024-08-21T02:11:35.7397141Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:35.7397690Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:35.7398618Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:35.7399406Z ##[endgroup] 2024-08-21T02:11:36.0721673Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-21T02:11:36.0722375Z with: 2024-08-21T02:11:36.0722667Z env: 2024-08-21T02:11:36.0722991Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:36.0723535Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:36.0724476Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:36.0725291Z ##[endgroup] 2024-08-21T02:11:36.0748250Z ##[group]Run set -eou pipefail 2024-08-21T02:11:36.0748793Z set -eou pipefail 2024-08-21T02:11:36.0749202Z  2024-08-21T02:11:36.0749821Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-21T02:11:36.0750604Z for _ in $(seq 1440); do 2024-08-21T02:11:36.0751168Z  # Break if no ssh session exists anymore 2024-08-21T02:11:36.0751772Z  if [ "$(who)" = "" ]; then 2024-08-21T02:11:36.0752251Z  break 2024-08-21T02:11:36.0752672Z  fi 2024-08-21T02:11:36.0753009Z  echo "." 2024-08-21T02:11:36.0753387Z  sleep 5 2024-08-21T02:11:36.0753751Z done 2024-08-21T02:11:36.0766712Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T02:11:36.0767332Z env: 2024-08-21T02:11:36.0767634Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:36.0768180Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:36.0769151Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:36.0769942Z ##[endgroup] 2024-08-21T02:11:36.0805717Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-21T02:11:36.0879849Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-21T02:11:36.0880820Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-21T02:11:36.0881563Z # shellcheck disable=SC2046 2024-08-21T02:11:36.0882120Z docker stop $(docker ps -q) || true 2024-08-21T02:11:36.0882895Z # Prune all of the docker images 2024-08-21T02:11:36.0883432Z docker system prune -af 2024-08-21T02:11:36.0895757Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T02:11:36.0896393Z env: 2024-08-21T02:11:36.0896714Z GIT_DEFAULT_BRANCH: main 2024-08-21T02:11:36.0897264Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-21T02:11:36.0898176Z DOCKER_CONTAINER_ID: cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:36.0899035Z ##[endgroup] 2024-08-21T02:11:36.4806215Z cb20aeb3a718 2024-08-21T02:11:37.4367924Z Deleted Containers: 2024-08-21T02:11:37.4368649Z cb20aeb3a71823161b0137771401292b81d822528731e4a01ce6be0f2082d917 2024-08-21T02:11:37.4369202Z 2024-08-21T02:11:46.3431660Z Deleted Images: 2024-08-21T02:11:46.3436226Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-21T02:11:46.3440252Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-21T02:11:46.3443619Z deleted: sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb 2024-08-21T02:11:46.3445922Z deleted: sha256:c2ce10014dd98f6abbfbf2265f9780a8204c2a7a27463e61ee565573a23d141d 2024-08-21T02:11:46.3447210Z deleted: sha256:ae249c0e6e6013bd6ffea2ca0b0160c015c9ebf7c03070bc2f983a9ff2415bf5 2024-08-21T02:11:46.3448383Z deleted: sha256:8a73883c1067c0e29ff128f553518682b47ed9707d2f35e7d5138e2caf029d61 2024-08-21T02:11:46.3450083Z deleted: sha256:9c040db729fa084584aaf5573082408a2196bc64e5952270c79dc610ed4680b6 2024-08-21T02:11:46.3451442Z deleted: sha256:17311eeeb8b71b16699880c0f646e185107496c4d41da5d23370603768ec030c 2024-08-21T02:11:46.3452849Z deleted: sha256:07cc10bece8501b1c8035d4385ad9e9867dd6e92fa32218a4dcb3e42af7e0411 2024-08-21T02:11:46.3454587Z deleted: sha256:f3e32d05099d1a3115a4722bc2e94a48752c3e965446773673808252671bb967 2024-08-21T02:11:46.3455768Z deleted: sha256:be7c3f41475e3e72ffe3ac2f92ab2f4cf49d5496b2aae738942ac3bb687d742c 2024-08-21T02:11:46.3456891Z deleted: sha256:4fcac490be825129b468cf458089f1fcf8fd56afc44ef4ab27c2eff41ab1ac90 2024-08-21T02:11:46.3458006Z deleted: sha256:31c66d80a2a4792897f267d363c0cf84fb821c5d4a275ab1636f778fd0284c08 2024-08-21T02:11:46.3459090Z deleted: sha256:8a8bcb6085e7e438e5533e8a5613e2ae5888543853568d023af3e2623a1593a9 2024-08-21T02:11:46.3460158Z deleted: sha256:2f6cb60029c79d11567b738f5bdbfb9cb1838cc0f194c4a5e6ae685178d2ed34 2024-08-21T02:11:46.3461225Z deleted: sha256:e546391804d3313f558e1ac16e93cb29a417a92e4eccaf0d4bd12af3bbcf39e3 2024-08-21T02:11:46.3462424Z deleted: sha256:1623b2aa16f2bdc6b572f6799498d9a92fa8114d07c440599b17aea72e22f8b1 2024-08-21T02:11:46.3463549Z deleted: sha256:c0386ca58b7cb7afee015b7e58d2f70525eaf73d6da2cc63868b20e14b1f9497 2024-08-21T02:11:46.3464671Z deleted: sha256:dcbf534ff289186e246531925fed90f72516eb3d63663bad234d611cdc025d62 2024-08-21T02:11:46.3465800Z deleted: sha256:0bab537af579f56c9cd51c7073fd53635fffb424b363236e0de38a7dd0e60ae8 2024-08-21T02:11:46.3466962Z deleted: sha256:d85a852ae1ee027e2fc17240ed68b87957b0a6d210c9119752d8979a0dcd55cf 2024-08-21T02:11:46.3468313Z deleted: sha256:f3821d761b555362e44cf9a1447d8fd2d95b37d54aea67c7519cd9e5db0c4d7e 2024-08-21T02:11:46.3469436Z deleted: sha256:c05e84a545db5dfcb0847c73d2b0f4749fa67aabce05705632ff7f76c84d4688 2024-08-21T02:11:46.3470539Z deleted: sha256:27054eae7ca67b2bd5ca0fd3f4488bf5c5f78e95487d8c45616f3d753d48a8b5 2024-08-21T02:11:46.3471619Z deleted: sha256:9804139f6ae97796aaa993fe4b89ea0cf111eb4d30a79f4c87c339c2e1e04fa0 2024-08-21T02:11:46.3472678Z deleted: sha256:ad6ae2fc594aaa359ff4e287323836570454eb24dd162930655e414d1d77c6dd 2024-08-21T02:11:46.3473764Z deleted: sha256:9cfbb36e328af52a248853a46aab88e266c09e2b44a03e0a42194a031b9b9453 2024-08-21T02:11:46.3475186Z deleted: sha256:a8a842bd7f37b8d50b6d12d6f95f46bbba57953db466fcbde9ba522625f024ef 2024-08-21T02:11:46.3477460Z deleted: sha256:da9738654b3c86f89ebeed2560f1bae093f1219f1eff90b0394b7f211ecabd04 2024-08-21T02:11:46.3479176Z deleted: sha256:cc683bf0cb245394e0f568984b2dd4fce1b6a3a7fe7c44734bae1aa1caaa3251 2024-08-21T02:11:46.3480275Z deleted: sha256:965f220e71c7d1ec19aaf41b1529ab287beb9d86eb8482c4c14094485ab34475 2024-08-21T02:11:46.3481403Z deleted: sha256:da32ecec41a73ce7dd29464f90c7951510a18cd378753659a78cd7fabd10dc36 2024-08-21T02:11:46.3482482Z deleted: sha256:d828d187cee9ad48dbc3702fb61d359619dd8a1ed582878680b51686bf117e5a 2024-08-21T02:11:46.3483589Z deleted: sha256:712111f4b12ff8e2787597d5fdcb043bd0824e55ddd97f262ef550506ca195f0 2024-08-21T02:11:46.3484670Z deleted: sha256:a064eea405958ad14dc6c794e5df9fd1d4515424272801ec559d45a59317309a 2024-08-21T02:11:46.3485821Z deleted: sha256:e124382d136bd5e9a8b409ca79c1f1ef76458deaf444b1b2060d3fa99dbeb506 2024-08-21T02:11:46.3486912Z deleted: sha256:6e10fe757cc190a1b31b47010f4f7610e02cb1b4b438f190dcd301aa4fe2278e 2024-08-21T02:11:46.3488010Z deleted: sha256:4cea352e10f46ca4729f96ac6c4600fc53d4c36e7658c2e0f19f4f309cd4a2a7 2024-08-21T02:11:46.3489137Z deleted: sha256:518bba00fe1d0933053066d21cac0235c9fb485abbc41557ec184b293095c924 2024-08-21T02:11:46.3490232Z deleted: sha256:a64f9ca42e9f27ff2690c15a5c84312e6efaccad45668d9b496cb542165cfb5a 2024-08-21T02:11:46.3491344Z deleted: sha256:096ea0fa14a6c420658f5276b8a8f5d264c0de7e18161ceda2cfc2f4a92cd9bf 2024-08-21T02:11:46.3492410Z deleted: sha256:6f91c94ce6e3f8034c91c66ee25c66a3d85dba9d813b749bd6d2000942a99edc 2024-08-21T02:11:46.3493502Z deleted: sha256:8f8d255eeae546cead8f9a94d237fa57a99940c0f5d51bc29a9d41c7df9cffd9 2024-08-21T02:11:46.3494644Z deleted: sha256:fe05276f4a49f91eef56bfbc3da87cfff5eb116a827fa2d30efe42ce1c134b40 2024-08-21T02:11:46.3495791Z deleted: sha256:5f83cd75d0669c813d4f58978c92e14305cd8f6a07eb9a60665a583460fa7b59 2024-08-21T02:11:46.3496871Z deleted: sha256:8f7ab6366169d39d8953129f650f1927d2fb1328b3f11e2fd9076adb5f46b93a 2024-08-21T02:11:46.3498152Z deleted: sha256:4146cfd5dba0f5308954749f98596b14a1d8f6d23bd93f7ad86c6c3d72239328 2024-08-21T02:11:46.3499295Z deleted: sha256:c6d83cb9ccc606ff63bb3c4d5a5e1ff1450c619b5aae494303a94ae25971cca0 2024-08-21T02:11:46.3500389Z deleted: sha256:71e037e0a3e0963c4565534a7387c082afcb04475d3ac629acf7a0ab2f49fadc 2024-08-21T02:11:46.3501436Z deleted: sha256:4b31135c564686bdaf23262b901237880aea635655721cc7e8c95dc42b05e06d 2024-08-21T02:11:46.3502487Z deleted: sha256:d09bd5c2f35cedcb846543941ee64ce1f684b52e4285696c93b3fb2032da8579 2024-08-21T02:11:46.3503638Z deleted: sha256:4de7fb7d4fc22212aae5fc1ceb637526916d0cced240d7c93d5edd6b0ea28013 2024-08-21T02:11:46.3504753Z deleted: sha256:69890e3e82746e10aff2f1084f11afb4b95a4a6c26b30f64b7001bc0a5fe32bf 2024-08-21T02:11:46.3505938Z deleted: sha256:0156df53b3224e645336ea52b4bb816f1e8d8c9b591748cd645e5e5649fa507b 2024-08-21T02:11:46.3506992Z deleted: sha256:0664624cb9cc4a386f31bf49919a8d8a11d9fea428d05585d9e6228f754bc7cd 2024-08-21T02:11:46.3508361Z deleted: sha256:8c2ddd5fa535c768c058a23f5cc4b6c64cdd71bb7458491282aabc7f7bb94d69 2024-08-21T02:11:46.3509468Z deleted: sha256:8dd03dbe0e1c5f47dd333074dff4bd00415f2f63727a65bf17866774e2ebbd0d 2024-08-21T02:11:46.3510555Z deleted: sha256:3c56b63e094bd18fad9da7a541d537210bca3b730c590d3641213259f6908d6f 2024-08-21T02:11:46.3511611Z deleted: sha256:51484fef07bd1266d4d6ff3d767751172660fc4cc6a1ffe883523a63c985e31e 2024-08-21T02:11:46.3512716Z deleted: sha256:03e83c94023982ab7dd8dc65a6df2024e342f1b760c91ec7a680ee45824869c7 2024-08-21T02:11:46.3513807Z deleted: sha256:58440ab3e8175b5de94a76485c8689a6dc186015475ebf4b386ebba309c5da34 2024-08-21T02:11:46.3514905Z deleted: sha256:9a980f3383e8aa511f0ef68982cb6dfc53b38d1ac8c3d673cb3ba3d45158c91e 2024-08-21T02:11:46.3516037Z deleted: sha256:cb0269c9cf1c47b6c4188ca7be4db261b5d3fc4b4a5153629e222dba7b1f9f98 2024-08-21T02:11:46.3517126Z deleted: sha256:41f1ba870cdffc05e0bf8d87b640fb9d4c153d7f4f3b775f37f94e9f3d584aad 2024-08-21T02:11:46.3518220Z deleted: sha256:7b502ab352f48e8b6eb5e6824a29f308952a5ee7c7d987562e1df7e47639d7f1 2024-08-21T02:11:46.3519499Z deleted: sha256:6ba5217e18738660e6ca8b7a7ec88845e70b0fd781301520bf46c71d99df3144 2024-08-21T02:11:46.3520566Z deleted: sha256:dd5a1c233594780163726bdbbf5de00f7b6b8dca1dace8a001340714c54baedb 2024-08-21T02:11:46.3521631Z deleted: sha256:c27f0db27da73e01f67f6cb78d10f355e350d829479ba4351d82055bb29f6597 2024-08-21T02:11:46.3522740Z deleted: sha256:4e3ce3c854f35c08be3dd95e0d6e718512a885e4614291d7d8b7cd23f48b7dd8 2024-08-21T02:11:46.3523832Z deleted: sha256:c2d6fb088bbec16e60cf0ad74b016f7287c92e659ec2db01271bc309b4dc6d83 2024-08-21T02:11:46.3524927Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-08-21T02:11:46.3526036Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-08-21T02:11:46.3527094Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-08-21T02:11:46.3528161Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-08-21T02:11:46.3529253Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-08-21T02:11:46.3530369Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-08-21T02:11:46.3531552Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-08-21T02:11:46.3532691Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-08-21T02:11:46.3533825Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-08-21T02:11:46.3534922Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-08-21T02:11:46.3536041Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-21T02:11:46.3536702Z 2024-08-21T02:11:46.3536860Z Total reclaimed space: 41.12GB 2024-08-21T02:11:46.3659908Z Post job cleanup. 2024-08-21T02:11:46.3734957Z Post job cleanup. 2024-08-21T02:11:46.4794659Z [command]/usr/bin/git version 2024-08-21T02:11:46.4838084Z git version 2.25.1 2024-08-21T02:11:46.4891914Z Temporarily overriding HOME='/home/weiwangmeta/actions-runner/_work/_temp/d20d4939-ff32-49c3-805c-28454433f519' before making global git config changes 2024-08-21T02:11:46.4893558Z Adding repository directory to the temporary git global config as a safe directory 2024-08-21T02:11:46.4896901Z [command]/usr/bin/git config --global --add safe.directory /home/weiwangmeta/actions-runner/_work/pytorch/pytorch 2024-08-21T02:11:46.4931989Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-21T02:11:46.4961685Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-21T02:11:46.5232072Z Entering 'android/libs/fbjni' 2024-08-21T02:11:46.5279700Z Entering 'third_party/FP16' 2024-08-21T02:11:46.5326082Z Entering 'third_party/FXdiv' 2024-08-21T02:11:46.5373593Z Entering 'third_party/NNPACK' 2024-08-21T02:11:46.5419124Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-21T02:11:46.5470211Z Entering 'third_party/XNNPACK' 2024-08-21T02:11:46.5544673Z Entering 'third_party/benchmark' 2024-08-21T02:11:46.5593430Z Entering 'third_party/cpp-httplib' 2024-08-21T02:11:46.5644259Z Entering 'third_party/cpuinfo' 2024-08-21T02:11:46.5693355Z Entering 'third_party/cudnn_frontend' 2024-08-21T02:11:46.5740025Z Entering 'third_party/cutlass' 2024-08-21T02:11:46.5799954Z Entering 'third_party/eigen' 2024-08-21T02:11:46.5849406Z Entering 'third_party/fbgemm' 2024-08-21T02:11:46.5896542Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-21T02:11:46.5941606Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-21T02:11:46.5986934Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-21T02:11:46.6039319Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-21T02:11:46.6088568Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-21T02:11:46.6134342Z Entering 'third_party/flatbuffers' 2024-08-21T02:11:46.6185276Z Entering 'third_party/fmt' 2024-08-21T02:11:46.6233087Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-21T02:11:46.6279790Z Entering 'third_party/gloo' 2024-08-21T02:11:46.6326224Z Entering 'third_party/googletest' 2024-08-21T02:11:46.6373107Z Entering 'third_party/ideep' 2024-08-21T02:11:46.6418217Z Entering 'third_party/ideep/mkl-dnn' 2024-08-21T02:11:46.6473354Z Entering 'third_party/ittapi' 2024-08-21T02:11:46.6520144Z Entering 'third_party/kineto' 2024-08-21T02:11:46.6567718Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-21T02:11:46.6612784Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-21T02:11:46.6659921Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-21T02:11:46.6706100Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-21T02:11:46.6750874Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-21T02:11:46.6796698Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-21T02:11:46.6845275Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-21T02:11:46.6891637Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-21T02:11:46.6936970Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-21T02:11:46.6983625Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-21T02:11:46.7030593Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-21T02:11:46.7076442Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-21T02:11:46.7123207Z Entering 'third_party/mimalloc' 2024-08-21T02:11:46.7172723Z Entering 'third_party/nccl/nccl' 2024-08-21T02:11:46.7220612Z Entering 'third_party/nlohmann' 2024-08-21T02:11:46.7267803Z Entering 'third_party/onnx' 2024-08-21T02:11:46.7334900Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-21T02:11:46.7383236Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-21T02:11:46.7436758Z Entering 'third_party/opentelemetry-cpp' 2024-08-21T02:11:46.7484973Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-21T02:11:46.7529595Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-21T02:11:46.7574609Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-21T02:11:46.7618784Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-21T02:11:46.7669433Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-21T02:11:46.7713427Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-21T02:11:46.7759484Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-21T02:11:46.7803526Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-21T02:11:46.7851558Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-21T02:11:46.7899643Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-21T02:11:46.7969731Z Entering 'third_party/pocketfft' 2024-08-21T02:11:46.8016241Z Entering 'third_party/protobuf' 2024-08-21T02:11:46.8066647Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-21T02:11:46.8112749Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-21T02:11:46.8159714Z Entering 'third_party/psimd' 2024-08-21T02:11:46.8205698Z Entering 'third_party/pthreadpool' 2024-08-21T02:11:46.8252095Z Entering 'third_party/pybind11' 2024-08-21T02:11:46.8299608Z Entering 'third_party/python-peachpy' 2024-08-21T02:11:46.8345869Z Entering 'third_party/sleef' 2024-08-21T02:11:46.8393510Z Entering 'third_party/tensorpipe' 2024-08-21T02:11:46.8440677Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-21T02:11:46.8485462Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-21T02:11:46.8913859Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-21T02:11:46.8963797Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-21T02:11:46.9007978Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-21T02:11:46.9075677Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-21T02:11:46.9096126Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9108217Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-21T02:11:46.9138408Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-21T02:11:46.9405805Z Entering 'android/libs/fbjni' 2024-08-21T02:11:46.9432858Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9470041Z Entering 'third_party/FP16' 2024-08-21T02:11:46.9496151Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9531702Z Entering 'third_party/FXdiv' 2024-08-21T02:11:46.9558898Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9593742Z Entering 'third_party/NNPACK' 2024-08-21T02:11:46.9619735Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9655755Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-21T02:11:46.9682640Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9717916Z Entering 'third_party/XNNPACK' 2024-08-21T02:11:46.9745201Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9802784Z Entering 'third_party/benchmark' 2024-08-21T02:11:46.9829731Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9864790Z Entering 'third_party/cpp-httplib' 2024-08-21T02:11:46.9891341Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9926943Z Entering 'third_party/cpuinfo' 2024-08-21T02:11:46.9954514Z http.https://github.com/.extraheader 2024-08-21T02:11:46.9989466Z Entering 'third_party/cudnn_frontend' 2024-08-21T02:11:47.0016439Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0051068Z Entering 'third_party/cutlass' 2024-08-21T02:11:47.0078094Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0122517Z Entering 'third_party/eigen' 2024-08-21T02:11:47.0148802Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0186517Z Entering 'third_party/fbgemm' 2024-08-21T02:11:47.0212697Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0248109Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-21T02:11:47.0274504Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0310271Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-21T02:11:47.0335626Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0371905Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-21T02:11:47.0397717Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0441661Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-21T02:11:47.0466999Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0502281Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-21T02:11:47.0527631Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0563363Z Entering 'third_party/flatbuffers' 2024-08-21T02:11:47.0590600Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0628606Z Entering 'third_party/fmt' 2024-08-21T02:11:47.0654640Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0688876Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-21T02:11:47.0716388Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0751779Z Entering 'third_party/gloo' 2024-08-21T02:11:47.0778225Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0812966Z Entering 'third_party/googletest' 2024-08-21T02:11:47.0839687Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0874354Z Entering 'third_party/ideep' 2024-08-21T02:11:47.0900634Z http.https://github.com/.extraheader 2024-08-21T02:11:47.0934754Z Entering 'third_party/ideep/mkl-dnn' 2024-08-21T02:11:47.0960142Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1004341Z Entering 'third_party/ittapi' 2024-08-21T02:11:47.1030544Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1064866Z Entering 'third_party/kineto' 2024-08-21T02:11:47.1092626Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1126938Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-21T02:11:47.1152875Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1188455Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-21T02:11:47.1213927Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1251644Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-21T02:11:47.1278036Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1313914Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-21T02:11:47.1338990Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1374930Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-21T02:11:47.1399925Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1434388Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-21T02:11:47.1460912Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1498551Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-21T02:11:47.1524827Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1561564Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-21T02:11:47.1588515Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1624330Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-21T02:11:47.1650331Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1687198Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-21T02:11:47.1714120Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1751592Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-21T02:11:47.1776800Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1813233Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-21T02:11:47.1839432Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1877031Z Entering 'third_party/mimalloc' 2024-08-21T02:11:47.1903518Z http.https://github.com/.extraheader 2024-08-21T02:11:47.1939743Z Entering 'third_party/nccl/nccl' 2024-08-21T02:11:47.1967376Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2003743Z Entering 'third_party/nlohmann' 2024-08-21T02:11:47.2032457Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2070470Z Entering 'third_party/onnx' 2024-08-21T02:11:47.2096940Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2152567Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-21T02:11:47.2180028Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2216119Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-21T02:11:47.2243341Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2283861Z Entering 'third_party/opentelemetry-cpp' 2024-08-21T02:11:47.2310233Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2348572Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-21T02:11:47.2375107Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2412028Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-21T02:11:47.2439155Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2474641Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-21T02:11:47.2500540Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2536592Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-21T02:11:47.2562664Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2600707Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-21T02:11:47.2625891Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2661268Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-21T02:11:47.2686619Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2722048Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-21T02:11:47.2747899Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2782298Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-21T02:11:47.2807453Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2845298Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-21T02:11:47.2872153Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2910531Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-21T02:11:47.2935831Z http.https://github.com/.extraheader 2024-08-21T02:11:47.2996353Z Entering 'third_party/pocketfft' 2024-08-21T02:11:47.3022798Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3058862Z Entering 'third_party/protobuf' 2024-08-21T02:11:47.3085299Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3125583Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-21T02:11:47.3151763Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3188066Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-21T02:11:47.3213427Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3250862Z Entering 'third_party/psimd' 2024-08-21T02:11:47.3277840Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3313974Z Entering 'third_party/pthreadpool' 2024-08-21T02:11:47.3340179Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3376045Z Entering 'third_party/pybind11' 2024-08-21T02:11:47.3402607Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3438353Z Entering 'third_party/python-peachpy' 2024-08-21T02:11:47.3463921Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3500475Z Entering 'third_party/sleef' 2024-08-21T02:11:47.3526745Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3562426Z Entering 'third_party/tensorpipe' 2024-08-21T02:11:47.3589199Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3624630Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-21T02:11:47.3649991Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3685275Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-21T02:11:47.3711526Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3745697Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-21T02:11:47.3770869Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3805342Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-21T02:11:47.3831286Z http.https://github.com/.extraheader 2024-08-21T02:11:47.3868682Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-21T02:11:47.3893724Z http.https://github.com/.extraheader 2024-08-21T02:11:47.4046664Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-21T02:11:47.4072889Z ##[group]Run '/home/weiwangmeta/post-job.sh' 2024-08-21T02:11:47.4085183Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-21T02:11:47.4085865Z ##[endgroup] 2024-08-21T02:11:47.4126061Z The approximate cost for this shard is below in $ 2024-08-21T02:11:47.4154602Z 3.203 2024-08-21T02:11:47.4386186Z Cleaning up orphan processes