2024-04-03T16:13:15.5061940Z Current runner version: '2.315.0' 2024-04-03T16:13:15.5071026Z Runner name: 'i-044537176b517f11b' 2024-04-03T16:13:15.5072121Z Runner group name: 'Default' 2024-04-03T16:13:15.5073340Z Machine name: 'ip-10-0-46-72' 2024-04-03T16:13:15.5079501Z ##[group]GITHUB_TOKEN Permissions 2024-04-03T16:13:15.5082372Z Actions: read 2024-04-03T16:13:15.5083059Z Checks: read 2024-04-03T16:13:15.5083723Z Contents: read 2024-04-03T16:13:15.5084476Z Deployments: read 2024-04-03T16:13:15.5085118Z Discussions: read 2024-04-03T16:13:15.5085802Z Issues: read 2024-04-03T16:13:15.5086525Z Metadata: read 2024-04-03T16:13:15.5087148Z Packages: read 2024-04-03T16:13:15.5087798Z Pages: read 2024-04-03T16:13:15.5088486Z PullRequests: read 2024-04-03T16:13:15.5089186Z RepositoryProjects: read 2024-04-03T16:13:15.5089960Z SecurityEvents: read 2024-04-03T16:13:15.5090656Z Statuses: read 2024-04-03T16:13:15.5091492Z ##[endgroup] 2024-04-03T16:13:15.5095965Z Secret source: None 2024-04-03T16:13:15.5097201Z Prepare workflow directory 2024-04-03T16:13:16.0118396Z Prepare all required actions 2024-04-03T16:13:16.0379278Z Getting action download info 2024-04-03T16:13:16.2386929Z Download action repository 'pytorch/test-infra@main' (SHA:f5543b9b2d7b7c7b8eccd6576b68fc0472b1fbde) 2024-04-03T16:13:16.6793756Z Download action repository 'pytorch/pytorch@main' (SHA:d895192e87f9a2c9ab3d6fab60f08855a08c3900) 2024-04-03T16:13:20.6686676Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-04-03T16:13:20.8005448Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-04-03T16:13:21.1760449Z Getting action download info 2024-04-03T16:13:21.2716068Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-04-03T16:13:21.4482096Z Getting action download info 2024-04-03T16:13:21.5462827Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-04-03T16:13:21.7614703Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/123154/merge (4302cb7ecb2b1ae758f931ad18aeedf6ae927cb0) 2024-04-03T16:13:21.7618508Z ##[group] Inputs 2024-04-03T16:13:21.7619094Z build-environment: linux-focal-cuda12.1-py3.10-gcc9 2024-04-03T16:13:21.7622466Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "deploy", "shard": 1, "num_shards": 1, "runner": "linux.4xlarge.nvidia.gpu"}]} 2024-04-03T16:13:21.7626508Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:13:21.7627857Z sync-tag: 2024-04-03T16:13:21.7629040Z timeout-minutes: 360 2024-04-03T16:13:21.7629471Z use-gha: 2024-04-03T16:13:21.7629821Z dashboard-tag: 2024-04-03T16:13:21.7630193Z s3-bucket: gha-artifacts 2024-04-03T16:13:21.7630632Z aws-role-to-assume: 2024-04-03T16:13:21.7631052Z ##[endgroup] 2024-04-03T16:13:21.7632286Z Complete job name: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) 2024-04-03T16:13:21.8840741Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-04-03T16:13:21.8841512Z with: 2024-04-03T16:13:21.8842328Z github-secret: *** 2024-04-03T16:13:21.8843614Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-04-03T16:13:21.8844982Z activate-with-label: false 2024-04-03T16:13:21.8845457Z label: with-ssh 2024-04-03T16:13:21.8846119Z remove-existing-keys: true 2024-04-03T16:13:21.8846613Z env: 2024-04-03T16:13:21.8846961Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:13:21.8847396Z ##[endgroup] 2024-04-03T16:13:22.0155548Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-04-03T16:13:22.2964523Z Grabbing public ssh keys from https://github.com/caic99.keys 2024-04-03T16:13:22.3779314Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-04-03T16:13:22.3801291Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-04-03T16:13:22.3846410Z Login using: ssh ec2-user@ec2-52-87-176-93.compute-1.amazonaws.com 2024-04-03T16:13:22.3847405Z All testing is done inside the container, to start an interactive session run: 2024-04-03T16:13:22.3848464Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-04-03T16:13:22.4187453Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-04-03T16:13:22.4188267Z with: 2024-04-03T16:13:22.4188679Z submodules: recursive 2024-04-03T16:13:22.4189165Z fetch-depth: 0 2024-04-03T16:13:22.4189565Z env: 2024-04-03T16:13:22.4189963Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:13:22.4190458Z ##[endgroup] 2024-04-03T16:13:22.4512743Z ##[group]Run retry () { 2024-04-03T16:13:22.4513345Z retry () { 2024-04-03T16:13:22.4514048Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-04-03T16:13:22.4514798Z } 2024-04-03T16:13:22.4515245Z echo "${GITHUB_WORKSPACE}" 2024-04-03T16:13:22.4515832Z if [ -z "${NO_SUDO}" ]; then 2024-04-03T16:13:22.4516485Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-04-03T16:13:22.4517124Z else 2024-04-03T16:13:22.4517617Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-04-03T16:13:22.4518201Z fi 2024-04-03T16:13:22.4518716Z mkdir "${GITHUB_WORKSPACE}" 2024-04-03T16:13:22.4549186Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:13:22.4549940Z env: 2024-04-03T16:13:22.4550338Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:13:22.4550835Z NO_SUDO: 2024-04-03T16:13:22.4551246Z ##[endgroup] 2024-04-03T16:13:22.4688280Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-04-03T16:13:24.6532264Z ##[group]Run malfet/checkout@silent-checkout 2024-04-03T16:13:24.6532890Z with: 2024-04-03T16:13:24.6533283Z ref: d253bd7befc64137de913dbb8727b511f1052411 2024-04-03T16:13:24.6533839Z fetch-depth: 0 2024-04-03T16:13:24.6534221Z submodules: recursive 2024-04-03T16:13:24.6534620Z quiet-checkout: true 2024-04-03T16:13:24.6535051Z repository: pytorch/pytorch 2024-04-03T16:13:24.6535663Z token: *** 2024-04-03T16:13:24.6535997Z ssh-strict: true 2024-04-03T16:13:24.6536393Z persist-credentials: true 2024-04-03T16:13:24.6536830Z clean: true 2024-04-03T16:13:24.6537225Z sparse-checkout-cone-mode: true 2024-04-03T16:13:24.6537703Z lfs: false 2024-04-03T16:13:24.6538065Z set-safe-directory: true 2024-04-03T16:13:24.6538471Z env: 2024-04-03T16:13:24.6538798Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:13:24.6539219Z ##[endgroup] 2024-04-03T16:13:24.7862427Z Syncing repository: pytorch/pytorch 2024-04-03T16:13:24.7865586Z ##[group]Getting Git version info 2024-04-03T16:13:24.7866507Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-04-03T16:13:24.7867725Z [command]/usr/bin/git version 2024-04-03T16:13:24.7868720Z git version 2.40.1 2024-04-03T16:13:24.7870713Z ##[endgroup] 2024-04-03T16:13:24.7887433Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/8973fd61-bd21-4b83-8b09-d04585db1f10' before making global git config changes 2024-04-03T16:13:24.7889060Z Adding repository directory to the temporary git global config as a safe directory 2024-04-03T16:13:24.7893722Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-04-03T16:13:24.7937932Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-04-03T16:13:24.7943958Z ##[group]Initializing the repository 2024-04-03T16:13:24.7948044Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-04-03T16:13:24.7979176Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-03T16:13:24.7980747Z hint: is subject to change. To configure the initial branch name to use in all 2024-04-03T16:13:24.7981831Z hint: of your new repositories, which will suppress this warning, call: 2024-04-03T16:13:24.7983313Z hint: 2024-04-03T16:13:24.7984051Z hint: git config --global init.defaultBranch 2024-04-03T16:13:24.7984622Z hint: 2024-04-03T16:13:24.7985264Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-03T16:13:24.7986335Z hint: 'development'. The just-created branch can be renamed via this command: 2024-04-03T16:13:24.7987467Z hint: 2024-04-03T16:13:24.7988372Z hint: git branch -m 2024-04-03T16:13:24.7989424Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-04-03T16:13:24.7992747Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-04-03T16:13:24.8026566Z ##[endgroup] 2024-04-03T16:13:24.8027272Z ##[group]Disabling automatic garbage collection 2024-04-03T16:13:24.8030512Z [command]/usr/bin/git config --local gc.auto 0 2024-04-03T16:13:24.8064544Z ##[endgroup] 2024-04-03T16:13:24.8065243Z ##[group]Setting up auth 2024-04-03T16:13:24.8074027Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-04-03T16:13:24.8109172Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-04-03T16:13:24.8412683Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-04-03T16:13:24.8442546Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-04-03T16:13:24.8735486Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-04-03T16:13:24.8781593Z ##[endgroup] 2024-04-03T16:13:24.8782407Z ##[group]Fetching the repository 2024-04-03T16:13:24.8790534Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-04-03T16:13:30.0778042Z remote: Enumerating objects: 1123747 2024-04-03T16:13:30.0779111Z remote: Enumerating objects: 1124411, done. 2024-04-03T16:13:30.0780752Z remote: Counting objects: 0% (1/664) 2024-04-03T16:13:30.0781838Z remote: Counting objects: 1% (7/664) 2024-04-03T16:13:30.0783194Z remote: Counting objects: 2% (14/664) 2024-04-03T16:13:30.0784510Z remote: Counting objects: 3% (20/664) 2024-04-03T16:13:30.0785485Z remote: Counting objects: 4% (27/664) 2024-04-03T16:13:30.0786109Z remote: Counting objects: 5% (34/664) 2024-04-03T16:13:30.0786742Z remote: Counting objects: 6% (40/664) 2024-04-03T16:13:30.0787375Z remote: Counting objects: 7% (47/664) 2024-04-03T16:13:30.0787973Z remote: Counting objects: 8% (54/664) 2024-04-03T16:13:30.0788597Z remote: Counting objects: 9% (60/664) 2024-04-03T16:13:30.0789199Z remote: Counting objects: 10% (67/664) 2024-04-03T16:13:30.0789789Z remote: Counting objects: 11% (74/664) 2024-04-03T16:13:30.0790397Z remote: Counting objects: 12% (80/664) 2024-04-03T16:13:30.0791025Z remote: Counting objects: 13% (87/664) 2024-04-03T16:13:30.0791627Z remote: Counting objects: 14% (93/664) 2024-04-03T16:13:30.0792240Z remote: Counting objects: 15% (100/664) 2024-04-03T16:13:30.0792882Z remote: Counting objects: 16% (107/664) 2024-04-03T16:13:30.0793849Z remote: Counting objects: 17% (113/664) 2024-04-03T16:13:30.0794475Z remote: Counting objects: 18% (120/664) 2024-04-03T16:13:30.0795121Z remote: Counting objects: 19% (127/664) 2024-04-03T16:13:30.0795714Z remote: Counting objects: 20% (133/664) 2024-04-03T16:13:30.0796330Z remote: Counting objects: 21% (140/664) 2024-04-03T16:13:30.0796946Z remote: Counting objects: 22% (147/664) 2024-04-03T16:13:30.0797551Z remote: Counting objects: 23% (153/664) 2024-04-03T16:13:30.0798148Z remote: Counting objects: 24% (160/664) 2024-04-03T16:13:30.0798761Z remote: Counting objects: 25% (166/664) 2024-04-03T16:13:30.0799991Z remote: Counting objects: 26% (173/664) 2024-04-03T16:13:30.0800564Z remote: Counting objects: 27% (180/664) 2024-04-03T16:13:30.0801370Z remote: Counting objects: 28% (186/664) 2024-04-03T16:13:30.0802016Z remote: Counting objects: 29% (193/664) 2024-04-03T16:13:30.0802625Z remote: Counting objects: 30% (200/664) 2024-04-03T16:13:30.0803238Z remote: Counting objects: 31% (206/664) 2024-04-03T16:13:30.0803854Z remote: Counting objects: 32% (213/664) 2024-04-03T16:13:30.0804453Z remote: Counting objects: 33% (220/664) 2024-04-03T16:13:30.0805154Z remote: Counting objects: 34% (226/664) 2024-04-03T16:13:30.0805774Z remote: Counting objects: 35% (233/664) 2024-04-03T16:13:30.0806367Z remote: Counting objects: 36% (240/664) 2024-04-03T16:13:30.0807065Z remote: Counting objects: 37% (246/664) 2024-04-03T16:13:30.0807682Z remote: Counting objects: 38% (253/664) 2024-04-03T16:13:30.0808337Z remote: Counting objects: 39% (259/664) 2024-04-03T16:13:30.0808894Z remote: Counting objects: 40% (266/664) 2024-04-03T16:13:30.0809514Z remote: Counting objects: 41% (273/664) 2024-04-03T16:13:30.0810129Z remote: Counting objects: 42% (279/664) 2024-04-03T16:13:30.0810769Z remote: Counting objects: 43% (286/664) 2024-04-03T16:13:30.0811383Z remote: Counting objects: 44% (293/664) 2024-04-03T16:13:30.0812001Z remote: Counting objects: 45% (299/664) 2024-04-03T16:13:30.0812599Z remote: Counting objects: 46% (306/664) 2024-04-03T16:13:30.0813216Z remote: Counting objects: 47% (313/664) 2024-04-03T16:13:30.0813832Z remote: Counting objects: 48% (319/664) 2024-04-03T16:13:30.0814449Z remote: Counting objects: 49% (326/664) 2024-04-03T16:13:30.0815047Z remote: Counting objects: 50% (332/664) 2024-04-03T16:13:30.0815659Z remote: Counting objects: 51% (339/664) 2024-04-03T16:13:30.0816274Z remote: Counting objects: 52% (346/664) 2024-04-03T16:13:30.0816870Z remote: Counting objects: 53% (352/664) 2024-04-03T16:13:30.0817489Z remote: Counting objects: 54% (359/664) 2024-04-03T16:13:30.0818108Z remote: Counting objects: 55% (366/664) 2024-04-03T16:13:30.0818714Z remote: Counting objects: 56% (372/664) 2024-04-03T16:13:30.0819466Z remote: Counting objects: 57% (379/664) 2024-04-03T16:13:30.0820210Z remote: Counting objects: 58% (386/664) 2024-04-03T16:13:30.0821000Z remote: Counting objects: 59% (392/664) 2024-04-03T16:13:30.0821752Z remote: Counting objects: 60% (399/664) 2024-04-03T16:13:30.0822485Z remote: Counting objects: 61% (406/664) 2024-04-03T16:13:30.0823207Z remote: Counting objects: 62% (412/664) 2024-04-03T16:13:30.0823972Z remote: Counting objects: 63% (419/664) 2024-04-03T16:13:30.0824716Z remote: Counting objects: 64% (425/664) 2024-04-03T16:13:30.0825518Z remote: Counting objects: 65% (432/664) 2024-04-03T16:13:30.0826199Z remote: Counting objects: 66% (439/664) 2024-04-03T16:13:30.0826982Z remote: Counting objects: 67% (445/664) 2024-04-03T16:13:30.0827768Z remote: Counting objects: 68% (452/664) 2024-04-03T16:13:30.0828454Z remote: Counting objects: 69% (459/664) 2024-04-03T16:13:30.0829349Z remote: Counting objects: 70% (465/664) 2024-04-03T16:13:30.0830123Z remote: Counting objects: 71% (472/664) 2024-04-03T16:13:30.0830843Z remote: Counting objects: 72% (479/664) 2024-04-03T16:13:30.0831585Z remote: Counting objects: 73% (485/664) 2024-04-03T16:13:30.0832351Z remote: Counting objects: 74% (492/664) 2024-04-03T16:13:30.0833041Z remote: Counting objects: 75% (498/664) 2024-04-03T16:13:30.0833782Z remote: Counting objects: 76% (505/664) 2024-04-03T16:13:30.0834545Z remote: Counting objects: 77% (512/664) 2024-04-03T16:13:30.0835272Z remote: Counting objects: 78% (518/664) 2024-04-03T16:13:30.0836006Z remote: Counting objects: 79% (525/664) 2024-04-03T16:13:30.0836786Z remote: Counting objects: 80% (532/664) 2024-04-03T16:13:30.0837562Z remote: Counting objects: 81% (538/664) 2024-04-03T16:13:30.0838308Z remote: Counting objects: 82% (545/664) 2024-04-03T16:13:30.0839346Z remote: Counting objects: 83% (552/664) 2024-04-03T16:13:30.0840056Z remote: Counting objects: 84% (558/664) 2024-04-03T16:13:30.0840793Z remote: Counting objects: 85% (565/664) 2024-04-03T16:13:30.0841579Z remote: Counting objects: 86% (572/664) 2024-04-03T16:13:30.0842308Z remote: Counting objects: 87% (578/664) 2024-04-03T16:13:30.0843000Z remote: Counting objects: 88% (585/664) 2024-04-03T16:13:30.0843813Z remote: Counting objects: 89% (591/664) 2024-04-03T16:13:30.0844549Z remote: Counting objects: 90% (598/664) 2024-04-03T16:13:30.0845227Z remote: Counting objects: 91% (605/664) 2024-04-03T16:13:30.0845996Z remote: Counting objects: 92% (611/664) 2024-04-03T16:13:30.0846735Z remote: Counting objects: 93% (618/664) 2024-04-03T16:13:30.0847424Z remote: Counting objects: 94% (625/664) 2024-04-03T16:13:30.0848235Z remote: Counting objects: 95% (631/664) 2024-04-03T16:13:30.0848984Z remote: Counting objects: 96% (638/664) 2024-04-03T16:13:30.0849657Z remote: Counting objects: 97% (645/664) 2024-04-03T16:13:30.0850440Z remote: Counting objects: 98% (651/664) 2024-04-03T16:13:30.0851171Z remote: Counting objects: 99% (658/664) 2024-04-03T16:13:30.0851847Z remote: Counting objects: 100% (664/664) 2024-04-03T16:13:30.0852717Z remote: Counting objects: 100% (664/664), done. 2024-04-03T16:13:30.0853527Z remote: Compressing objects: 0% (1/369) 2024-04-03T16:13:30.0885767Z remote: Compressing objects: 1% (4/369) 2024-04-03T16:13:30.0947479Z remote: Compressing objects: 2% (8/369) 2024-04-03T16:13:30.1079233Z remote: Compressing objects: 3% (12/369) 2024-04-03T16:13:30.1147118Z remote: Compressing objects: 4% (15/369) 2024-04-03T16:13:30.1484695Z remote: Compressing objects: 5% (19/369) 2024-04-03T16:13:30.1558225Z remote: Compressing objects: 6% (23/369) 2024-04-03T16:13:30.1677343Z remote: Compressing objects: 7% (26/369) 2024-04-03T16:13:30.1796550Z remote: Compressing objects: 8% (30/369) 2024-04-03T16:13:30.1821691Z remote: Compressing objects: 9% (34/369) 2024-04-03T16:13:30.1870841Z remote: Compressing objects: 10% (37/369) 2024-04-03T16:13:30.1953790Z remote: Compressing objects: 11% (41/369) 2024-04-03T16:13:30.1963584Z remote: Compressing objects: 12% (45/369) 2024-04-03T16:13:30.2036863Z remote: Compressing objects: 13% (48/369) 2024-04-03T16:13:30.2037693Z remote: Compressing objects: 14% (52/369) 2024-04-03T16:13:30.2038428Z remote: Compressing objects: 15% (56/369) 2024-04-03T16:13:30.2070757Z remote: Compressing objects: 16% (60/369) 2024-04-03T16:13:30.2094129Z remote: Compressing objects: 17% (63/369) 2024-04-03T16:13:30.2114092Z remote: Compressing objects: 18% (67/369) 2024-04-03T16:13:30.2114744Z remote: Compressing objects: 19% (71/369) 2024-04-03T16:13:30.2115614Z remote: Compressing objects: 20% (74/369) 2024-04-03T16:13:30.2116281Z remote: Compressing objects: 21% (78/369) 2024-04-03T16:13:30.2116922Z remote: Compressing objects: 22% (82/369) 2024-04-03T16:13:30.2117580Z remote: Compressing objects: 23% (85/369) 2024-04-03T16:13:30.2118237Z remote: Compressing objects: 24% (89/369) 2024-04-03T16:13:30.2118894Z remote: Compressing objects: 25% (93/369) 2024-04-03T16:13:30.2119797Z remote: Compressing objects: 26% (96/369) 2024-04-03T16:13:30.2123231Z remote: Compressing objects: 27% (100/369) 2024-04-03T16:13:30.2123908Z remote: Compressing objects: 28% (104/369) 2024-04-03T16:13:30.2133030Z remote: Compressing objects: 29% (108/369) 2024-04-03T16:13:30.2133839Z remote: Compressing objects: 30% (111/369) 2024-04-03T16:13:30.2134537Z remote: Compressing objects: 31% (115/369) 2024-04-03T16:13:30.2135196Z remote: Compressing objects: 32% (119/369) 2024-04-03T16:13:30.2143045Z remote: Compressing objects: 33% (122/369) 2024-04-03T16:13:30.2154817Z remote: Compressing objects: 34% (126/369) 2024-04-03T16:13:30.2166021Z remote: Compressing objects: 35% (130/369) 2024-04-03T16:13:30.2166644Z remote: Compressing objects: 36% (133/369) 2024-04-03T16:13:30.2187369Z remote: Compressing objects: 37% (137/369) 2024-04-03T16:13:30.2188465Z remote: Compressing objects: 38% (141/369) 2024-04-03T16:13:30.2189057Z remote: Compressing objects: 39% (144/369) 2024-04-03T16:13:30.2189715Z remote: Compressing objects: 40% (148/369) 2024-04-03T16:13:30.2190379Z remote: Compressing objects: 41% (152/369) 2024-04-03T16:13:30.2191037Z remote: Compressing objects: 42% (155/369) 2024-04-03T16:13:30.2197652Z remote: Compressing objects: 43% (159/369) 2024-04-03T16:13:30.2198343Z remote: Compressing objects: 44% (163/369) 2024-04-03T16:13:30.2198954Z remote: Compressing objects: 45% (167/369) 2024-04-03T16:13:30.2199824Z remote: Compressing objects: 46% (170/369) 2024-04-03T16:13:30.2200493Z remote: Compressing objects: 47% (174/369) 2024-04-03T16:13:30.2201145Z remote: Compressing objects: 48% (178/369) 2024-04-03T16:13:30.2211339Z remote: Compressing objects: 49% (181/369) 2024-04-03T16:13:30.2212044Z remote: Compressing objects: 50% (185/369) 2024-04-03T16:13:30.2212707Z remote: Compressing objects: 51% (189/369) 2024-04-03T16:13:30.2213321Z remote: Compressing objects: 52% (192/369) 2024-04-03T16:13:30.2213964Z remote: Compressing objects: 53% (196/369) 2024-04-03T16:13:30.2214616Z remote: Compressing objects: 54% (200/369) 2024-04-03T16:13:30.2215284Z remote: Compressing objects: 55% (203/369) 2024-04-03T16:13:30.2215932Z remote: Compressing objects: 56% (207/369) 2024-04-03T16:13:30.2216590Z remote: Compressing objects: 57% (211/369) 2024-04-03T16:13:30.2217273Z remote: Compressing objects: 58% (215/369) 2024-04-03T16:13:30.2217910Z remote: Compressing objects: 59% (218/369) 2024-04-03T16:13:30.2218568Z remote: Compressing objects: 60% (222/369) 2024-04-03T16:13:30.2219229Z remote: Compressing objects: 61% (226/369) 2024-04-03T16:13:30.2219866Z remote: Compressing objects: 62% (229/369) 2024-04-03T16:13:30.2220525Z remote: Compressing objects: 63% (233/369) 2024-04-03T16:13:30.2221183Z remote: Compressing objects: 64% (237/369) 2024-04-03T16:13:30.2221822Z remote: Compressing objects: 65% (240/369) 2024-04-03T16:13:30.2222482Z remote: Compressing objects: 66% (244/369) 2024-04-03T16:13:30.2223183Z remote: Compressing objects: 67% (248/369) 2024-04-03T16:13:30.2223820Z remote: Compressing objects: 68% (251/369) 2024-04-03T16:13:30.2224466Z remote: Compressing objects: 69% (255/369) 2024-04-03T16:13:30.2225335Z remote: Compressing objects: 70% (259/369) 2024-04-03T16:13:30.2225993Z remote: Compressing objects: 71% (262/369) 2024-04-03T16:13:30.2226630Z remote: Compressing objects: 72% (266/369) 2024-04-03T16:13:30.2227282Z remote: Compressing objects: 73% (270/369) 2024-04-03T16:13:30.2227940Z remote: Compressing objects: 74% (274/369) 2024-04-03T16:13:30.2235671Z remote: Compressing objects: 75% (277/369) 2024-04-03T16:13:30.2236329Z remote: Compressing objects: 76% (281/369) 2024-04-03T16:13:30.2236979Z remote: Compressing objects: 77% (285/369) 2024-04-03T16:13:30.2237629Z remote: Compressing objects: 78% (288/369) 2024-04-03T16:13:30.2238274Z remote: Compressing objects: 79% (292/369) 2024-04-03T16:13:30.2238931Z remote: Compressing objects: 80% (296/369) 2024-04-03T16:13:30.2240014Z remote: Compressing objects: 81% (299/369) 2024-04-03T16:13:30.2240677Z remote: Compressing objects: 82% (303/369) 2024-04-03T16:13:30.2241344Z remote: Compressing objects: 83% (307/369) 2024-04-03T16:13:30.2242000Z remote: Compressing objects: 84% (310/369) 2024-04-03T16:13:30.2242643Z remote: Compressing objects: 85% (314/369) 2024-04-03T16:13:30.2259612Z remote: Compressing objects: 86% (318/369) 2024-04-03T16:13:30.2260272Z remote: Compressing objects: 87% (322/369) 2024-04-03T16:13:30.2261052Z remote: Compressing objects: 88% (325/369) 2024-04-03T16:13:30.2262087Z remote: Compressing objects: 89% (329/369) 2024-04-03T16:13:30.2262807Z remote: Compressing objects: 90% (333/369) 2024-04-03T16:13:30.2263664Z remote: Compressing objects: 91% (336/369) 2024-04-03T16:13:30.2264305Z remote: Compressing objects: 92% (340/369) 2024-04-03T16:13:30.2264972Z remote: Compressing objects: 93% (344/369) 2024-04-03T16:13:30.2265632Z remote: Compressing objects: 94% (347/369) 2024-04-03T16:13:30.2266288Z remote: Compressing objects: 95% (351/369) 2024-04-03T16:13:30.2266952Z remote: Compressing objects: 96% (355/369) 2024-04-03T16:13:30.2267684Z remote: Compressing objects: 97% (358/369) 2024-04-03T16:13:30.2268651Z remote: Compressing objects: 98% (362/369) 2024-04-03T16:13:30.2269309Z remote: Compressing objects: 99% (366/369) 2024-04-03T16:13:30.2269965Z remote: Compressing objects: 100% (369/369) 2024-04-03T16:13:30.2270657Z remote: Compressing objects: 100% (369/369), done. 2024-04-03T16:13:57.3094995Z remote: Total 1124411 (delta 364), reused 532 (delta 295), pack-reused 1123747 2024-04-03T16:14:27.5836357Z [command]/usr/bin/git rev-parse --verify --quiet d253bd7befc64137de913dbb8727b511f1052411^{object} 2024-04-03T16:14:27.5865470Z d253bd7befc64137de913dbb8727b511f1052411 2024-04-03T16:14:27.5871212Z ##[endgroup] 2024-04-03T16:14:27.5871916Z ##[group]Determining the checkout info 2024-04-03T16:14:27.5873886Z ##[endgroup] 2024-04-03T16:14:27.5876398Z ##[group]Checking out the ref 2024-04-03T16:14:27.5878779Z [command]/usr/bin/git checkout --quiet --force d253bd7befc64137de913dbb8727b511f1052411 2024-04-03T16:14:29.5276928Z ##[endgroup] 2024-04-03T16:14:29.5277854Z ##[group]Setting up auth for fetching submodules 2024-04-03T16:14:29.5284694Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-04-03T16:14:29.5338475Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-04-03T16:14:29.5370729Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-04-03T16:14:29.5404790Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-04-03T16:14:29.5434647Z ##[endgroup] 2024-04-03T16:14:29.5435307Z ##[group]Fetching submodules 2024-04-03T16:14:29.5441146Z [command]/usr/bin/git submodule sync --recursive 2024-04-03T16:14:29.5775708Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-04-03T16:14:29.6092624Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-04-03T16:14:29.6094779Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-04-03T16:14:29.6098414Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-04-03T16:14:29.6101698Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-04-03T16:14:29.6105034Z Submodule 'third_party/QNNPACK' (https://github.com/pytorch/QNNPACK) registered for path 'third_party/QNNPACK' 2024-04-03T16:14:29.6109363Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-04-03T16:14:29.6112043Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-04-03T16:14:29.6116364Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-04-03T16:14:29.6120772Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-04-03T16:14:29.6125922Z Submodule 'third_party/cub' (https://github.com/NVlabs/cub.git) registered for path 'third_party/cub' 2024-04-03T16:14:29.6130292Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-04-03T16:14:29.6134541Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-04-03T16:14:29.6138977Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-04-03T16:14:29.6143614Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-04-03T16:14:29.6148560Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-04-03T16:14:29.6153323Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-04-03T16:14:29.6158372Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-04-03T16:14:29.6164625Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-04-03T16:14:29.6169611Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-04-03T16:14:29.6175228Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-04-03T16:14:29.6180669Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-04-03T16:14:29.6186601Z Submodule 'third_party/ios-cmake' (https://github.com/Yangqing/ios-cmake.git) registered for path 'third_party/ios-cmake' 2024-04-03T16:14:29.6192350Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-04-03T16:14:29.6198323Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-04-03T16:14:29.6205100Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-04-03T16:14:29.6211183Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-04-03T16:14:29.6217597Z Submodule 'third_party/neon2sse' (https://github.com/intel/ARM_NEON_2_x86_SSE.git) registered for path 'third_party/neon2sse' 2024-04-03T16:14:29.6224009Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-04-03T16:14:29.6230502Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-04-03T16:14:29.6237397Z Submodule 'third_party/onnx-tensorrt' (https://github.com/onnx/onnx-tensorrt) registered for path 'third_party/onnx-tensorrt' 2024-04-03T16:14:29.6244871Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-04-03T16:14:29.6251840Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-04-03T16:14:29.6258962Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-04-03T16:14:29.6266578Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-04-03T16:14:29.6273666Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-04-03T16:14:29.6282000Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-04-03T16:14:29.6289567Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-04-03T16:14:29.6297254Z Submodule 'third_party/tbb' (https://github.com/01org/tbb) registered for path 'third_party/tbb' 2024-04-03T16:14:29.6305454Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-04-03T16:14:29.6313819Z Submodule 'third_party/zstd' (https://github.com/facebook/zstd.git) registered for path 'third_party/zstd' 2024-04-03T16:14:29.6343316Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-04-03T16:14:29.9097685Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-04-03T16:14:30.0987632Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-04-03T16:14:30.2747634Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-04-03T16:14:30.5203299Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/QNNPACK'... 2024-04-03T16:14:30.7875740Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-04-03T16:14:32.9924237Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-04-03T16:14:45.7668760Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-04-03T16:14:46.3027776Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-04-03T16:14:47.0013487Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cub'... 2024-04-03T16:14:48.6302877Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-04-03T16:14:50.1134097Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-04-03T16:14:53.0075442Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-04-03T16:15:00.0439452Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-04-03T16:15:02.5720783Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-04-03T16:15:04.4788589Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-04-03T16:15:05.9038675Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-04-03T16:15:06.0806707Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-04-03T16:15:06.5536335Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-04-03T16:15:06.9154562Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-04-03T16:15:08.2296829Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-04-03T16:15:08.6127842Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ios-cmake'... 2024-04-03T16:15:08.8435183Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-04-03T16:15:09.2044363Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-04-03T16:15:11.0299276Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-04-03T16:15:11.7925273Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-04-03T16:15:12.4174430Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/neon2sse'... 2024-04-03T16:15:12.7100215Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-04-03T16:15:20.2971372Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-04-03T16:15:23.0352895Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx-tensorrt'... 2024-04-03T16:15:23.5486326Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-04-03T16:15:23.7638041Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-04-03T16:15:33.5319405Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-04-03T16:15:33.7004368Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-04-03T16:15:33.8947395Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-04-03T16:15:34.9177698Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-04-03T16:15:35.2182781Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-04-03T16:15:35.8850216Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tbb'... 2024-04-03T16:15:38.2756205Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-04-03T16:15:38.7196568Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/zstd'... 2024-04-03T16:15:41.5697095Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-04-03T16:15:41.5827168Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-04-03T16:15:41.5925734Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-04-03T16:15:41.6213076Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-04-03T16:15:41.6496557Z Submodule path 'third_party/QNNPACK': checked out '7d2a4e9931a82adc3814275b6219a03e24e36b4c' 2024-04-03T16:15:41.6939721Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-04-03T16:15:42.8720229Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-04-03T16:15:42.8978406Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-04-03T16:15:43.0150696Z Submodule path 'third_party/cpuinfo': checked out 'd6860c477c99f1fce9e28eb206891af3c0e1a1d7' 2024-04-03T16:15:43.0572210Z Submodule path 'third_party/cub': checked out 'd106ddb991a56c3df1b6d51b2409e36ba8181ce4' 2024-04-03T16:15:43.0903009Z Submodule path 'third_party/cudnn_frontend': checked out '150798fe976556078f443fdb059a1ff0361f58a2' 2024-04-03T16:15:43.7560549Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-04-03T16:15:44.0692008Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-04-03T16:15:44.1518142Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-04-03T16:15:44.1538046Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-04-03T16:15:44.1543234Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-04-03T16:15:44.1545420Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-04-03T16:15:44.1548203Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-04-03T16:15:44.1552307Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-04-03T16:15:44.1579356Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-04-03T16:15:45.2930826Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-04-03T16:15:45.9471759Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-04-03T16:15:48.8099220Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-04-03T16:15:50.0826395Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-04-03T16:15:50.5233522Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-04-03T16:15:50.6427861Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-04-03T16:15:51.1824723Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-04-03T16:15:51.2582198Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-04-03T16:15:51.2709671Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-04-03T16:15:51.4103022Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-04-03T16:15:51.4531553Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-04-03T16:15:51.4632264Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-04-03T16:15:51.5104003Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-04-03T16:15:51.5396411Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-04-03T16:15:51.5923688Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-04-03T16:15:51.6064058Z Submodule path 'third_party/ideep': checked out '8a6cc4e09dc509f04f83c085e38786b1fb44e14d' 2024-04-03T16:15:51.6082849Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-04-03T16:15:51.6107968Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-04-03T16:16:05.1264646Z Submodule path 'third_party/ideep/mkl-dnn': checked out '86e6af5974177e513fd3fee58425e1063e7f1361' 2024-04-03T16:16:05.1384734Z Submodule path 'third_party/ios-cmake': checked out '8abaed637d56f1337d6e1d2c4026e25c1eade724' 2024-04-03T16:16:05.1577126Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-04-03T16:16:05.2774311Z Submodule path 'third_party/kineto': checked out '8466a8b111b36dc725e6855d52a0b133d925a8e0' 2024-04-03T16:16:05.2794375Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-04-03T16:16:05.2799720Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-04-03T16:16:05.2802118Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-04-03T16:16:05.2827145Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-04-03T16:16:06.2625334Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-04-03T16:16:07.6802186Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-04-03T16:16:09.0749958Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-04-03T16:16:09.0767455Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-04-03T16:16:09.0770949Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-04-03T16:16:09.0774673Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-04-03T16:16:09.0778620Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-04-03T16:16:09.0782542Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-04-03T16:16:09.0786814Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-04-03T16:16:09.0790963Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-04-03T16:16:09.0795461Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-04-03T16:16:09.0823683Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-04-03T16:16:10.4186600Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-04-03T16:16:10.8297691Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-04-03T16:16:12.3016199Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-04-03T16:16:12.6517826Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-04-03T16:16:13.0510996Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-04-03T16:16:14.3377761Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-04-03T16:16:21.8626738Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-04-03T16:16:22.2782632Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-04-03T16:16:22.2998190Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-04-03T16:16:22.3465962Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-04-03T16:16:22.3617604Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-04-03T16:16:22.3635306Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-04-03T16:16:22.3661273Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-04-03T16:16:23.1180023Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-04-03T16:16:23.1386095Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-04-03T16:16:23.1866783Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-04-03T16:16:23.3153429Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-04-03T16:16:23.3338169Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-04-03T16:16:23.3780964Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-04-03T16:16:23.4480324Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-04-03T16:16:23.4899196Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-04-03T16:16:23.5174304Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-04-03T16:16:23.5334393Z Submodule path 'third_party/neon2sse': checked out '97a126f08ce318023be604d03f88bf0820a9464a' 2024-04-03T16:16:23.6675755Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-04-03T16:16:24.0867944Z Submodule path 'third_party/onnx': checked out 'ccde5da81388ffa770ca98b64e07f803ad089414' 2024-04-03T16:16:24.0904311Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-04-03T16:16:24.0907009Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-04-03T16:16:24.0934804Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-04-03T16:16:24.5130396Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-04-03T16:16:25.6523106Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-04-03T16:16:25.6920118Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-04-03T16:16:25.7126961Z Submodule path 'third_party/onnx-tensorrt': checked out 'c153211418a7c57ce071d9ce2a41f8d1c85a878f' 2024-04-03T16:16:25.7143649Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx-tensorrt/third_party/onnx' 2024-04-03T16:16:25.7170706Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx-tensorrt/third_party/onnx'... 2024-04-03T16:16:28.4532215Z Submodule path 'third_party/onnx-tensorrt/third_party/onnx': checked out '765f5ee823a67a866f4bd28a9860e81f3c811ce8' 2024-04-03T16:16:28.4554846Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark' 2024-04-03T16:16:28.4557822Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11' 2024-04-03T16:16:28.4586013Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark'... 2024-04-03T16:16:28.9061779Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11'... 2024-04-03T16:16:29.9624646Z Submodule path 'third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark': checked out 'e776aa0275e293707b6a0901e0e8d8a8a3679508' 2024-04-03T16:16:30.0485515Z Submodule path 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11': checked out 'a1041190c8b8ff0cd9e2f0752248ad5e3789ea0c' 2024-04-03T16:16:30.0501639Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang' 2024-04-03T16:16:30.0528872Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang'... 2024-04-03T16:16:30.2361268Z Submodule path 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-04-03T16:16:30.2465199Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-04-03T16:16:30.5670733Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-04-03T16:16:30.5693854Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-04-03T16:16:30.5696590Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-04-03T16:16:30.5723995Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-04-03T16:16:31.0406325Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-04-03T16:16:32.3364825Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-04-03T16:16:32.4227868Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-04-03T16:16:32.4322359Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-04-03T16:16:32.4458315Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-04-03T16:16:32.4851578Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-04-03T16:16:32.5186973Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-04-03T16:16:32.5680571Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-04-03T16:16:32.7106139Z Submodule path 'third_party/tbb': checked out 'a51a90bc609bb73db8ea13841b5cf7aa4344d4a9' 2024-04-03T16:16:32.7416659Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-04-03T16:16:32.7438732Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-04-03T16:16:32.7443420Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-04-03T16:16:32.7447426Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-04-03T16:16:32.7452111Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-04-03T16:16:32.7478817Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-04-03T16:16:34.0810245Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-04-03T16:16:34.3016783Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-04-03T16:16:35.6575144Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-04-03T16:16:36.7358641Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-04-03T16:16:36.7529729Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-04-03T16:16:36.8355389Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-04-03T16:16:36.8690650Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-04-03T16:16:36.8706371Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-04-03T16:16:36.8733523Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-04-03T16:16:37.0663145Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-04-03T16:16:37.2428338Z Submodule path 'third_party/zstd': checked out 'aec56a52fbab207fc639a1937d1e708a282edca8' 2024-04-03T16:16:37.2463503Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-04-03T16:16:37.2796717Z Entering 'android/libs/fbjni' 2024-04-03T16:16:37.2838808Z Entering 'third_party/FP16' 2024-04-03T16:16:37.2881596Z Entering 'third_party/FXdiv' 2024-04-03T16:16:37.2923866Z Entering 'third_party/NNPACK' 2024-04-03T16:16:37.2967276Z Entering 'third_party/QNNPACK' 2024-04-03T16:16:37.3011198Z Entering 'third_party/VulkanMemoryAllocator' 2024-04-03T16:16:37.3053983Z Entering 'third_party/XNNPACK' 2024-04-03T16:16:37.3115668Z Entering 'third_party/benchmark' 2024-04-03T16:16:37.3158275Z Entering 'third_party/cpuinfo' 2024-04-03T16:16:37.3201615Z Entering 'third_party/cub' 2024-04-03T16:16:37.3244931Z Entering 'third_party/cudnn_frontend' 2024-04-03T16:16:37.3288116Z Entering 'third_party/cutlass' 2024-04-03T16:16:37.3338462Z Entering 'third_party/eigen' 2024-04-03T16:16:37.3384611Z Entering 'third_party/fbgemm' 2024-04-03T16:16:37.3427386Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-04-03T16:16:37.3468228Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-04-03T16:16:37.3510217Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-04-03T16:16:37.3559044Z Entering 'third_party/fbgemm/third_party/googletest' 2024-04-03T16:16:37.3599999Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-04-03T16:16:37.3643040Z Entering 'third_party/flatbuffers' 2024-04-03T16:16:37.3688898Z Entering 'third_party/fmt' 2024-04-03T16:16:37.3731144Z Entering 'third_party/foxi' 2024-04-03T16:16:37.3772998Z Entering 'third_party/gemmlowp/gemmlowp' 2024-04-03T16:16:37.3814561Z Entering 'third_party/gloo' 2024-04-03T16:16:37.3856198Z Entering 'third_party/googletest' 2024-04-03T16:16:37.3898409Z Entering 'third_party/ideep' 2024-04-03T16:16:37.3939426Z Entering 'third_party/ideep/mkl-dnn' 2024-04-03T16:16:37.3988506Z Entering 'third_party/ios-cmake' 2024-04-03T16:16:37.4030217Z Entering 'third_party/ittapi' 2024-04-03T16:16:37.4072674Z Entering 'third_party/kineto' 2024-04-03T16:16:37.4114149Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-04-03T16:16:37.4155323Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-04-03T16:16:37.4198045Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-04-03T16:16:37.4239178Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-04-03T16:16:37.4281301Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-04-03T16:16:37.4320899Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-04-03T16:16:37.4363344Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-04-03T16:16:37.4404845Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-04-03T16:16:37.4446638Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-04-03T16:16:37.4490135Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-04-03T16:16:37.4532198Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-04-03T16:16:37.4571878Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-04-03T16:16:37.4613603Z Entering 'third_party/mimalloc' 2024-04-03T16:16:37.4656341Z Entering 'third_party/nccl/nccl' 2024-04-03T16:16:37.4698033Z Entering 'third_party/neon2sse' 2024-04-03T16:16:37.4739490Z Entering 'third_party/nlohmann' 2024-04-03T16:16:37.4782686Z Entering 'third_party/onnx' 2024-04-03T16:16:37.4842300Z Entering 'third_party/onnx/third_party/benchmark' 2024-04-03T16:16:37.4883215Z Entering 'third_party/onnx/third_party/pybind11' 2024-04-03T16:16:37.4927740Z Entering 'third_party/onnx-tensorrt' 2024-04-03T16:16:37.4967905Z Entering 'third_party/onnx-tensorrt/third_party/onnx' 2024-04-03T16:16:37.5013465Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark' 2024-04-03T16:16:37.5054915Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11' 2024-04-03T16:16:37.5095318Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang' 2024-04-03T16:16:37.5140875Z Entering 'third_party/pocketfft' 2024-04-03T16:16:37.5182347Z Entering 'third_party/protobuf' 2024-04-03T16:16:37.5227226Z Entering 'third_party/protobuf/third_party/benchmark' 2024-04-03T16:16:37.5267718Z Entering 'third_party/protobuf/third_party/googletest' 2024-04-03T16:16:37.5310316Z Entering 'third_party/psimd' 2024-04-03T16:16:37.5351710Z Entering 'third_party/pthreadpool' 2024-04-03T16:16:37.5393485Z Entering 'third_party/pybind11' 2024-04-03T16:16:37.5435177Z Entering 'third_party/python-peachpy' 2024-04-03T16:16:37.5477032Z Entering 'third_party/sleef' 2024-04-03T16:16:37.5518585Z Entering 'third_party/tbb' 2024-04-03T16:16:37.5563529Z Entering 'third_party/tensorpipe' 2024-04-03T16:16:37.5605202Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-04-03T16:16:37.5646015Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-04-03T16:16:37.5686860Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-04-03T16:16:37.5727360Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-04-03T16:16:37.5767420Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-04-03T16:16:37.5811928Z Entering 'third_party/zstd' 2024-04-03T16:16:37.5866859Z ##[endgroup] 2024-04-03T16:16:37.5869365Z ##[group]Persisting credentials for submodules 2024-04-03T16:16:37.5873974Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-04-03T16:16:37.6192260Z Entering 'android/libs/fbjni' 2024-04-03T16:16:37.6245260Z Entering 'third_party/FP16' 2024-04-03T16:16:37.6296075Z Entering 'third_party/FXdiv' 2024-04-03T16:16:37.6348715Z Entering 'third_party/NNPACK' 2024-04-03T16:16:37.6401010Z Entering 'third_party/QNNPACK' 2024-04-03T16:16:37.6453671Z Entering 'third_party/VulkanMemoryAllocator' 2024-04-03T16:16:37.6507117Z Entering 'third_party/XNNPACK' 2024-04-03T16:16:37.6578382Z Entering 'third_party/benchmark' 2024-04-03T16:16:37.6632008Z Entering 'third_party/cpuinfo' 2024-04-03T16:16:37.6685526Z Entering 'third_party/cub' 2024-04-03T16:16:37.6737052Z Entering 'third_party/cudnn_frontend' 2024-04-03T16:16:37.6787860Z Entering 'third_party/cutlass' 2024-04-03T16:16:37.6847381Z Entering 'third_party/eigen' 2024-04-03T16:16:37.6901242Z Entering 'third_party/fbgemm' 2024-04-03T16:16:37.6952760Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-04-03T16:16:37.7003675Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-04-03T16:16:37.7054001Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-04-03T16:16:37.7110237Z Entering 'third_party/fbgemm/third_party/googletest' 2024-04-03T16:16:37.7160525Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-04-03T16:16:37.7211982Z Entering 'third_party/flatbuffers' 2024-04-03T16:16:37.7266491Z Entering 'third_party/fmt' 2024-04-03T16:16:37.7320152Z Entering 'third_party/foxi' 2024-04-03T16:16:37.7371329Z Entering 'third_party/gemmlowp/gemmlowp' 2024-04-03T16:16:37.7423998Z Entering 'third_party/gloo' 2024-04-03T16:16:37.7477093Z Entering 'third_party/googletest' 2024-04-03T16:16:37.7530802Z Entering 'third_party/ideep' 2024-04-03T16:16:37.7580965Z Entering 'third_party/ideep/mkl-dnn' 2024-04-03T16:16:37.7638317Z Entering 'third_party/ios-cmake' 2024-04-03T16:16:37.7690667Z Entering 'third_party/ittapi' 2024-04-03T16:16:37.7742077Z Entering 'third_party/kineto' 2024-04-03T16:16:37.7793464Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-04-03T16:16:37.7845287Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-04-03T16:16:37.7896647Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-04-03T16:16:37.7948376Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-04-03T16:16:37.7999552Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-04-03T16:16:37.8050787Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-04-03T16:16:37.8104459Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-04-03T16:16:37.8155746Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-04-03T16:16:37.8207300Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-04-03T16:16:37.8259037Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-04-03T16:16:37.8311733Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-04-03T16:16:37.8363506Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-04-03T16:16:37.8415277Z Entering 'third_party/mimalloc' 2024-04-03T16:16:37.8467581Z Entering 'third_party/nccl/nccl' 2024-04-03T16:16:37.8520660Z Entering 'third_party/neon2sse' 2024-04-03T16:16:37.8571881Z Entering 'third_party/nlohmann' 2024-04-03T16:16:37.8624387Z Entering 'third_party/onnx' 2024-04-03T16:16:37.8693659Z Entering 'third_party/onnx/third_party/benchmark' 2024-04-03T16:16:37.8744500Z Entering 'third_party/onnx/third_party/pybind11' 2024-04-03T16:16:37.8797766Z Entering 'third_party/onnx-tensorrt' 2024-04-03T16:16:37.8848530Z Entering 'third_party/onnx-tensorrt/third_party/onnx' 2024-04-03T16:16:37.8903905Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark' 2024-04-03T16:16:37.8955442Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11' 2024-04-03T16:16:37.9006673Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang' 2024-04-03T16:16:37.9062579Z Entering 'third_party/pocketfft' 2024-04-03T16:16:37.9113313Z Entering 'third_party/protobuf' 2024-04-03T16:16:37.9169222Z Entering 'third_party/protobuf/third_party/benchmark' 2024-04-03T16:16:37.9219738Z Entering 'third_party/protobuf/third_party/googletest' 2024-04-03T16:16:37.9272519Z Entering 'third_party/psimd' 2024-04-03T16:16:37.9325148Z Entering 'third_party/pthreadpool' 2024-04-03T16:16:37.9376344Z Entering 'third_party/pybind11' 2024-04-03T16:16:37.9428383Z Entering 'third_party/python-peachpy' 2024-04-03T16:16:37.9479658Z Entering 'third_party/sleef' 2024-04-03T16:16:37.9531799Z Entering 'third_party/tbb' 2024-04-03T16:16:37.9585038Z Entering 'third_party/tensorpipe' 2024-04-03T16:16:37.9635445Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-04-03T16:16:37.9686701Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-04-03T16:16:37.9736932Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-04-03T16:16:37.9787220Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-04-03T16:16:37.9836506Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-04-03T16:16:37.9890929Z Entering 'third_party/zstd' 2024-04-03T16:16:37.9959166Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-04-03T16:16:38.0290252Z Entering 'android/libs/fbjni' 2024-04-03T16:16:38.0338358Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-04-03T16:16:38.0357066Z Entering 'third_party/FP16' 2024-04-03T16:16:38.0406346Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-04-03T16:16:38.0425365Z Entering 'third_party/FXdiv' 2024-04-03T16:16:38.0473530Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-04-03T16:16:38.0492433Z Entering 'third_party/NNPACK' 2024-04-03T16:16:38.0541184Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-04-03T16:16:38.0561623Z Entering 'third_party/QNNPACK' 2024-04-03T16:16:38.0609249Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/QNNPACK/config remote.origin.url 2024-04-03T16:16:38.0629153Z Entering 'third_party/VulkanMemoryAllocator' 2024-04-03T16:16:38.0677115Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-04-03T16:16:38.0696079Z Entering 'third_party/XNNPACK' 2024-04-03T16:16:38.0744699Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-04-03T16:16:38.0783443Z Entering 'third_party/benchmark' 2024-04-03T16:16:38.0831592Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-04-03T16:16:38.0850828Z Entering 'third_party/cpuinfo' 2024-04-03T16:16:38.0900065Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-04-03T16:16:38.0919550Z Entering 'third_party/cub' 2024-04-03T16:16:38.0967340Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cub/config remote.origin.url 2024-04-03T16:16:38.0987308Z Entering 'third_party/cudnn_frontend' 2024-04-03T16:16:38.1036183Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-04-03T16:16:38.1055314Z Entering 'third_party/cutlass' 2024-04-03T16:16:38.1102709Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-04-03T16:16:38.1130893Z Entering 'third_party/eigen' 2024-04-03T16:16:38.1178094Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-04-03T16:16:38.1199249Z Entering 'third_party/fbgemm' 2024-04-03T16:16:38.1246452Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-04-03T16:16:38.1264763Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-04-03T16:16:38.1312269Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-04-03T16:16:38.1331723Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-04-03T16:16:38.1377793Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-04-03T16:16:38.1396567Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-04-03T16:16:38.1444850Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-04-03T16:16:38.1469326Z Entering 'third_party/fbgemm/third_party/googletest' 2024-04-03T16:16:38.1518268Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-04-03T16:16:38.1535497Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-04-03T16:16:38.1583256Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-04-03T16:16:38.1602520Z Entering 'third_party/flatbuffers' 2024-04-03T16:16:38.1650784Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-04-03T16:16:38.1671919Z Entering 'third_party/fmt' 2024-04-03T16:16:38.1720404Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-04-03T16:16:38.1739052Z Entering 'third_party/foxi' 2024-04-03T16:16:38.1788241Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-04-03T16:16:38.1807606Z Entering 'third_party/gemmlowp/gemmlowp' 2024-04-03T16:16:38.1854686Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-04-03T16:16:38.1873896Z Entering 'third_party/gloo' 2024-04-03T16:16:38.1922036Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-04-03T16:16:38.1941032Z Entering 'third_party/googletest' 2024-04-03T16:16:38.1988534Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-04-03T16:16:38.2008183Z Entering 'third_party/ideep' 2024-04-03T16:16:38.2055161Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-04-03T16:16:38.2073066Z Entering 'third_party/ideep/mkl-dnn' 2024-04-03T16:16:38.2120851Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-04-03T16:16:38.2146251Z Entering 'third_party/ios-cmake' 2024-04-03T16:16:38.2193278Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ios-cmake/config remote.origin.url 2024-04-03T16:16:38.2213229Z Entering 'third_party/ittapi' 2024-04-03T16:16:38.2259533Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-04-03T16:16:38.2278551Z Entering 'third_party/kineto' 2024-04-03T16:16:38.2325500Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-04-03T16:16:38.2344619Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-04-03T16:16:38.2392424Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-04-03T16:16:38.2411632Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-04-03T16:16:38.2459643Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-04-03T16:16:38.2481004Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-04-03T16:16:38.2529300Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-04-03T16:16:38.2548633Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-04-03T16:16:38.2595814Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-04-03T16:16:38.2615991Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-04-03T16:16:38.2663588Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-04-03T16:16:38.2681782Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-04-03T16:16:38.2729174Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-04-03T16:16:38.2748785Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-04-03T16:16:38.2796324Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-04-03T16:16:38.2816223Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-04-03T16:16:38.2863367Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-04-03T16:16:38.2883036Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-04-03T16:16:38.2930421Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-04-03T16:16:38.2951057Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-04-03T16:16:38.2998335Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-04-03T16:16:38.3018891Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-04-03T16:16:38.3067891Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-04-03T16:16:38.3087288Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-04-03T16:16:38.3135066Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-04-03T16:16:38.3155686Z Entering 'third_party/mimalloc' 2024-04-03T16:16:38.3204139Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-04-03T16:16:38.3223043Z Entering 'third_party/nccl/nccl' 2024-04-03T16:16:38.3271110Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-04-03T16:16:38.3291164Z Entering 'third_party/neon2sse' 2024-04-03T16:16:38.3338851Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/neon2sse/config remote.origin.url 2024-04-03T16:16:38.3357413Z Entering 'third_party/nlohmann' 2024-04-03T16:16:38.3406651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-04-03T16:16:38.3426076Z Entering 'third_party/onnx' 2024-04-03T16:16:38.3474005Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-04-03T16:16:38.3511182Z Entering 'third_party/onnx/third_party/benchmark' 2024-04-03T16:16:38.3561316Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-04-03T16:16:38.3581234Z Entering 'third_party/onnx/third_party/pybind11' 2024-04-03T16:16:38.3629321Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-04-03T16:16:38.3651575Z Entering 'third_party/onnx-tensorrt' 2024-04-03T16:16:38.3701401Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx-tensorrt/config remote.origin.url 2024-04-03T16:16:38.3719094Z Entering 'third_party/onnx-tensorrt/third_party/onnx' 2024-04-03T16:16:38.3768683Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx-tensorrt/modules/third_party/onnx/config remote.origin.url 2024-04-03T16:16:38.3792069Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark' 2024-04-03T16:16:38.3841027Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx-tensorrt/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-04-03T16:16:38.3859790Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11' 2024-04-03T16:16:38.3909484Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx-tensorrt/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-04-03T16:16:38.3927733Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang' 2024-04-03T16:16:38.3976723Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx-tensorrt/modules/third_party/onnx/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-04-03T16:16:38.4001288Z Entering 'third_party/pocketfft' 2024-04-03T16:16:38.4049444Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-04-03T16:16:38.4068667Z Entering 'third_party/protobuf' 2024-04-03T16:16:38.4117280Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-04-03T16:16:38.4140971Z Entering 'third_party/protobuf/third_party/benchmark' 2024-04-03T16:16:38.4190087Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-04-03T16:16:38.4209631Z Entering 'third_party/protobuf/third_party/googletest' 2024-04-03T16:16:38.4258325Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-04-03T16:16:38.4278757Z Entering 'third_party/psimd' 2024-04-03T16:16:38.4328335Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-04-03T16:16:38.4346889Z Entering 'third_party/pthreadpool' 2024-04-03T16:16:38.4395898Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-04-03T16:16:38.4415021Z Entering 'third_party/pybind11' 2024-04-03T16:16:38.4463595Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-04-03T16:16:38.4483501Z Entering 'third_party/python-peachpy' 2024-04-03T16:16:38.4532432Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-04-03T16:16:38.4551852Z Entering 'third_party/sleef' 2024-04-03T16:16:38.4601031Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-04-03T16:16:38.4619523Z Entering 'third_party/tbb' 2024-04-03T16:16:38.4668478Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tbb/config remote.origin.url 2024-04-03T16:16:38.4691317Z Entering 'third_party/tensorpipe' 2024-04-03T16:16:38.4838730Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-04-03T16:16:38.4856785Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-04-03T16:16:38.4907196Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-04-03T16:16:38.4925634Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-04-03T16:16:38.4974011Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-04-03T16:16:38.4991334Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-04-03T16:16:38.5041467Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-04-03T16:16:38.5060636Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-04-03T16:16:38.5109930Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-04-03T16:16:38.5127401Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-04-03T16:16:38.5175561Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-04-03T16:16:38.5197271Z Entering 'third_party/zstd' 2024-04-03T16:16:38.5247769Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/zstd/config remote.origin.url 2024-04-03T16:16:38.6301291Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-04-03T16:16:38.6628697Z Entering 'android/libs/fbjni' 2024-04-03T16:16:38.6671435Z Entering 'third_party/FP16' 2024-04-03T16:16:38.6714519Z Entering 'third_party/FXdiv' 2024-04-03T16:16:38.6757878Z Entering 'third_party/NNPACK' 2024-04-03T16:16:38.6801913Z Entering 'third_party/QNNPACK' 2024-04-03T16:16:38.6844338Z Entering 'third_party/VulkanMemoryAllocator' 2024-04-03T16:16:38.6887542Z Entering 'third_party/XNNPACK' 2024-04-03T16:16:38.6949309Z Entering 'third_party/benchmark' 2024-04-03T16:16:38.6991568Z Entering 'third_party/cpuinfo' 2024-04-03T16:16:38.7035169Z Entering 'third_party/cub' 2024-04-03T16:16:38.7079813Z Entering 'third_party/cudnn_frontend' 2024-04-03T16:16:38.7121384Z Entering 'third_party/cutlass' 2024-04-03T16:16:38.7171754Z Entering 'third_party/eigen' 2024-04-03T16:16:38.7216132Z Entering 'third_party/fbgemm' 2024-04-03T16:16:38.7259490Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-04-03T16:16:38.7300339Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-04-03T16:16:38.7343213Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-04-03T16:16:38.7390816Z Entering 'third_party/fbgemm/third_party/googletest' 2024-04-03T16:16:38.7432754Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-04-03T16:16:38.7474752Z Entering 'third_party/flatbuffers' 2024-04-03T16:16:38.7520603Z Entering 'third_party/fmt' 2024-04-03T16:16:38.7563152Z Entering 'third_party/foxi' 2024-04-03T16:16:38.7604776Z Entering 'third_party/gemmlowp/gemmlowp' 2024-04-03T16:16:38.7646858Z Entering 'third_party/gloo' 2024-04-03T16:16:38.7689994Z Entering 'third_party/googletest' 2024-04-03T16:16:38.7732436Z Entering 'third_party/ideep' 2024-04-03T16:16:38.7772606Z Entering 'third_party/ideep/mkl-dnn' 2024-04-03T16:16:38.7821348Z Entering 'third_party/ios-cmake' 2024-04-03T16:16:38.7864250Z Entering 'third_party/ittapi' 2024-04-03T16:16:38.7906784Z Entering 'third_party/kineto' 2024-04-03T16:16:38.7949023Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-04-03T16:16:38.7990324Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-04-03T16:16:38.8033452Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-04-03T16:16:38.8073838Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-04-03T16:16:38.8115507Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-04-03T16:16:38.8155110Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-04-03T16:16:38.8199591Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-04-03T16:16:38.8240889Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-04-03T16:16:38.8282382Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-04-03T16:16:38.8323958Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-04-03T16:16:38.8368560Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-04-03T16:16:38.8408739Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-04-03T16:16:38.8450999Z Entering 'third_party/mimalloc' 2024-04-03T16:16:38.8493627Z Entering 'third_party/nccl/nccl' 2024-04-03T16:16:38.8536460Z Entering 'third_party/neon2sse' 2024-04-03T16:16:38.8578649Z Entering 'third_party/nlohmann' 2024-04-03T16:16:38.8622412Z Entering 'third_party/onnx' 2024-04-03T16:16:38.8681609Z Entering 'third_party/onnx/third_party/benchmark' 2024-04-03T16:16:38.8723624Z Entering 'third_party/onnx/third_party/pybind11' 2024-04-03T16:16:38.8768099Z Entering 'third_party/onnx-tensorrt' 2024-04-03T16:16:38.8809918Z Entering 'third_party/onnx-tensorrt/third_party/onnx' 2024-04-03T16:16:38.8855120Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark' 2024-04-03T16:16:38.8896602Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11' 2024-04-03T16:16:38.8936451Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang' 2024-04-03T16:16:38.8982898Z Entering 'third_party/pocketfft' 2024-04-03T16:16:38.9025867Z Entering 'third_party/protobuf' 2024-04-03T16:16:38.9071712Z Entering 'third_party/protobuf/third_party/benchmark' 2024-04-03T16:16:38.9111682Z Entering 'third_party/protobuf/third_party/googletest' 2024-04-03T16:16:38.9155918Z Entering 'third_party/psimd' 2024-04-03T16:16:38.9198306Z Entering 'third_party/pthreadpool' 2024-04-03T16:16:38.9241006Z Entering 'third_party/pybind11' 2024-04-03T16:16:38.9283814Z Entering 'third_party/python-peachpy' 2024-04-03T16:16:38.9326259Z Entering 'third_party/sleef' 2024-04-03T16:16:38.9369112Z Entering 'third_party/tbb' 2024-04-03T16:16:38.9414715Z Entering 'third_party/tensorpipe' 2024-04-03T16:16:38.9455809Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-04-03T16:16:38.9495575Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-04-03T16:16:38.9536930Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-04-03T16:16:38.9577455Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-04-03T16:16:38.9616873Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-04-03T16:16:38.9661761Z Entering 'third_party/zstd' 2024-04-03T16:16:38.9720909Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-04-03T16:16:39.0040016Z Entering 'android/libs/fbjni' 2024-04-03T16:16:39.0082538Z Entering 'third_party/FP16' 2024-04-03T16:16:39.0123611Z Entering 'third_party/FXdiv' 2024-04-03T16:16:39.0166209Z Entering 'third_party/NNPACK' 2024-04-03T16:16:39.0208723Z Entering 'third_party/QNNPACK' 2024-04-03T16:16:39.0250459Z Entering 'third_party/VulkanMemoryAllocator' 2024-04-03T16:16:39.0292643Z Entering 'third_party/XNNPACK' 2024-04-03T16:16:39.0352994Z Entering 'third_party/benchmark' 2024-04-03T16:16:39.0395407Z Entering 'third_party/cpuinfo' 2024-04-03T16:16:39.0439739Z Entering 'third_party/cub' 2024-04-03T16:16:39.0482034Z Entering 'third_party/cudnn_frontend' 2024-04-03T16:16:39.0524542Z Entering 'third_party/cutlass' 2024-04-03T16:16:39.0575037Z Entering 'third_party/eigen' 2024-04-03T16:16:39.0619552Z Entering 'third_party/fbgemm' 2024-04-03T16:16:39.0661287Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-04-03T16:16:39.0703286Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-04-03T16:16:39.0745791Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-04-03T16:16:39.0794567Z Entering 'third_party/fbgemm/third_party/googletest' 2024-04-03T16:16:39.0835159Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-04-03T16:16:39.0877091Z Entering 'third_party/flatbuffers' 2024-04-03T16:16:39.0922451Z Entering 'third_party/fmt' 2024-04-03T16:16:39.0964236Z Entering 'third_party/foxi' 2024-04-03T16:16:39.1005658Z Entering 'third_party/gemmlowp/gemmlowp' 2024-04-03T16:16:39.1048019Z Entering 'third_party/gloo' 2024-04-03T16:16:39.1090591Z Entering 'third_party/googletest' 2024-04-03T16:16:39.1131532Z Entering 'third_party/ideep' 2024-04-03T16:16:39.1172109Z Entering 'third_party/ideep/mkl-dnn' 2024-04-03T16:16:39.1218963Z Entering 'third_party/ios-cmake' 2024-04-03T16:16:39.1261781Z Entering 'third_party/ittapi' 2024-04-03T16:16:39.1303257Z Entering 'third_party/kineto' 2024-04-03T16:16:39.1345949Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-04-03T16:16:39.1388011Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-04-03T16:16:39.1431527Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-04-03T16:16:39.1472131Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-04-03T16:16:39.1513778Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-04-03T16:16:39.1556391Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-04-03T16:16:39.1600649Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-04-03T16:16:39.1643393Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-04-03T16:16:39.1685035Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-04-03T16:16:39.1728833Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-04-03T16:16:39.1773880Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-04-03T16:16:39.1815076Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-04-03T16:16:39.1857655Z Entering 'third_party/mimalloc' 2024-04-03T16:16:39.1900554Z Entering 'third_party/nccl/nccl' 2024-04-03T16:16:39.1943447Z Entering 'third_party/neon2sse' 2024-04-03T16:16:39.1985240Z Entering 'third_party/nlohmann' 2024-04-03T16:16:39.2027923Z Entering 'third_party/onnx' 2024-04-03T16:16:39.2087299Z Entering 'third_party/onnx/third_party/benchmark' 2024-04-03T16:16:39.2128664Z Entering 'third_party/onnx/third_party/pybind11' 2024-04-03T16:16:39.2172002Z Entering 'third_party/onnx-tensorrt' 2024-04-03T16:16:39.2212935Z Entering 'third_party/onnx-tensorrt/third_party/onnx' 2024-04-03T16:16:39.2260069Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark' 2024-04-03T16:16:39.2301107Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11' 2024-04-03T16:16:39.2341685Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang' 2024-04-03T16:16:39.2388277Z Entering 'third_party/pocketfft' 2024-04-03T16:16:39.2430559Z Entering 'third_party/protobuf' 2024-04-03T16:16:39.2475439Z Entering 'third_party/protobuf/third_party/benchmark' 2024-04-03T16:16:39.2516092Z Entering 'third_party/protobuf/third_party/googletest' 2024-04-03T16:16:39.2559714Z Entering 'third_party/psimd' 2024-04-03T16:16:39.2600946Z Entering 'third_party/pthreadpool' 2024-04-03T16:16:39.2642823Z Entering 'third_party/pybind11' 2024-04-03T16:16:39.2685240Z Entering 'third_party/python-peachpy' 2024-04-03T16:16:39.2727190Z Entering 'third_party/sleef' 2024-04-03T16:16:39.2768552Z Entering 'third_party/tbb' 2024-04-03T16:16:39.2812675Z Entering 'third_party/tensorpipe' 2024-04-03T16:16:39.2853698Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-04-03T16:16:39.2893806Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-04-03T16:16:39.2933903Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-04-03T16:16:39.2973756Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-04-03T16:16:39.3012886Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-04-03T16:16:39.3057584Z Entering 'third_party/zstd' 2024-04-03T16:16:39.3150402Z ##[endgroup] 2024-04-03T16:16:39.3153709Z [command]/usr/bin/git log -1 --format='%H' 2024-04-03T16:16:39.3181947Z 'd253bd7befc64137de913dbb8727b511f1052411' 2024-04-03T16:16:39.3398518Z Prepare all required actions 2024-04-03T16:16:39.3399747Z Getting action download info 2024-04-03T16:16:39.4778697Z ##[group]Run ./.github/actions/setup-linux 2024-04-03T16:16:39.4779224Z env: 2024-04-03T16:16:39.4779559Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:39.4779965Z ##[endgroup] 2024-04-03T16:16:39.4862454Z ##[group]Run set -euo pipefail 2024-04-03T16:16:39.4862971Z set -euo pipefail 2024-04-03T16:16:39.4863455Z function get_ec2_metadata() { 2024-04-03T16:16:39.4864111Z  # Pulled from instance metadata endpoint for EC2 2024-04-03T16:16:39.4865229Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-04-03T16:16:39.4866231Z  category=$1 2024-04-03T16:16:39.4866884Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-04-03T16:16:39.4867657Z  runner_name_str=i-044537176b517f11b 2024-04-03T16:16:39.4868358Z  if [[ $runner_name_str != *"gcp"* ]]; then 2024-04-03T16:16:39.4869249Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-04-03T16:16:39.4869976Z  else 2024-04-03T16:16:39.4870620Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-04-03T16:16:39.4871382Z  fi 2024-04-03T16:16:39.4871710Z } 2024-04-03T16:16:39.4872151Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-04-03T16:16:39.4872881Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-04-03T16:16:39.4873681Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-04-03T16:16:39.4874448Z echo "system info $(uname -a)" 2024-04-03T16:16:39.4883806Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:39.4884430Z env: 2024-04-03T16:16:39.4884746Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:39.4885170Z ##[endgroup] 2024-04-03T16:16:39.4988727Z ami-id: ami-0ce0c36d7a00b20e2 2024-04-03T16:16:39.5054489Z instance-id: i-044537176b517f11b 2024-04-03T16:16:39.5119540Z instance-type: g3.4xlarge 2024-04-03T16:16:39.5129049Z system info Linux ip-10-0-46-72.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-04-03T16:16:39.5156887Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-04-03T16:16:39.5158055Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-04-03T16:16:39.5166743Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:39.5167366Z env: 2024-04-03T16:16:39.5167683Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:39.5168112Z ##[endgroup] 2024-04-03T16:16:39.5255883Z ##[group]Run if systemctl is-active --quiet docker; then 2024-04-03T16:16:39.5256702Z if systemctl is-active --quiet docker; then 2024-04-03T16:16:39.5257446Z  echo "Docker daemon is running..."; 2024-04-03T16:16:39.5258078Z else 2024-04-03T16:16:39.5258745Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-04-03T16:16:39.5259544Z fi 2024-04-03T16:16:39.5267741Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:39.5268429Z env: 2024-04-03T16:16:39.5268885Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:39.5269386Z ##[endgroup] 2024-04-03T16:16:39.5316973Z Docker daemon is running... 2024-04-03T16:16:39.5375320Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-04-03T16:16:39.5376021Z with: 2024-04-03T16:16:39.5376348Z shell: bash 2024-04-03T16:16:39.5376690Z timeout_minutes: 5 2024-04-03T16:16:39.5377083Z max_attempts: 3 2024-04-03T16:16:39.5377658Z retry_wait_seconds: 30 2024-04-03T16:16:39.5379566Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-04-03T16:16:39.5381414Z polling_interval_seconds: 1 2024-04-03T16:16:39.5381874Z warning_on_retry: true 2024-04-03T16:16:39.5382299Z continue_on_error: false 2024-04-03T16:16:39.5382689Z env: 2024-04-03T16:16:39.5383015Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:39.5383449Z AWS_RETRY_MODE: standard 2024-04-03T16:16:39.5383856Z AWS_MAX_ATTEMPTS: 5 2024-04-03T16:16:39.5384268Z AWS_DEFAULT_REGION: us-east-1 2024-04-03T16:16:39.5384721Z ##[endgroup] 2024-04-03T16:16:40.5515488Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-04-03T16:16:40.5516384Z Configure a credential helper to remove this warning. See 2024-04-03T16:16:40.5517489Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-04-03T16:16:40.5518160Z 2024-04-03T16:16:40.5518305Z Login Succeeded 2024-04-03T16:16:40.6100896Z Command completed after 1 attempt(s). 2024-04-03T16:16:40.6155931Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-04-03T16:16:40.6156837Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-04-03T16:16:40.6157655Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-04-03T16:16:40.6167079Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:40.6167708Z env: 2024-04-03T16:16:40.6168028Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:40.6168463Z ##[endgroup] 2024-04-03T16:16:40.6249002Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-04-03T16:16:40.6250046Z # ignore expansion of "docker ps -q" since it could be empty 2024-04-03T16:16:40.6250858Z # shellcheck disable=SC2046 2024-04-03T16:16:40.6251516Z docker stop $(docker ps -q) || true 2024-04-03T16:16:40.6252157Z # Prune all of the docker images 2024-04-03T16:16:40.6252785Z docker system prune -af 2024-04-03T16:16:40.6261239Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:40.6261938Z env: 2024-04-03T16:16:40.6262329Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:40.6262822Z ##[endgroup] 2024-04-03T16:16:40.6675971Z "docker stop" requires at least 1 argument. 2024-04-03T16:16:40.6677498Z See 'docker stop --help'. 2024-04-03T16:16:40.6678235Z 2024-04-03T16:16:40.6678846Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-04-03T16:16:40.6680175Z 2024-04-03T16:16:40.6680604Z Stop one or more running containers 2024-04-03T16:16:40.7109063Z Deleted Images: 2024-04-03T16:16:40.7110557Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/tool/alpine:latest 2024-04-03T16:16:40.7112678Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/tool/alpine@sha256:def822f9851ca422481ec6fee59a9966f12b351c62ccb9aca841526ffaa9f748 2024-04-03T16:16:40.7114299Z deleted: sha256:6dbb9cc54074106d46d4ccb330f2a40a682d49dda5f4844962b7dce9fe44aaec 2024-04-03T16:16:40.7115483Z deleted: sha256:b2d5eeeaba3a22b9b8aa97261957974a6bd65274ebd43e1d81d0a7b8b752b116 2024-04-03T16:16:40.7116024Z 2024-04-03T16:16:40.7125818Z Total reclaimed space: 5.613MB 2024-04-03T16:16:40.7167231Z ##[group]Run set +e 2024-04-03T16:16:40.7167695Z set +e 2024-04-03T16:16:40.7168049Z set -x 2024-04-03T16:16:40.7168408Z  2024-04-03T16:16:40.7168860Z PT_DOMAIN=download.pytorch.org 2024-04-03T16:16:40.7169840Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-04-03T16:16:40.7171211Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-04-03T16:16:40.7172158Z # one is returned at random 2024-04-03T16:16:40.7172814Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-04-03T16:16:40.7173789Z  2024-04-03T16:16:40.7174169Z if [ -z "${RESOLVED_IP}" ]; then 2024-04-03T16:16:40.7174975Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-04-03T16:16:40.7175891Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-04-03T16:16:40.7176606Z  2024-04-03T16:16:40.7177005Z  if [ -z "${RESOLVED_IP}" ]; then 2024-04-03T16:16:40.7177676Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-04-03T16:16:40.7178313Z  exit 1 2024-04-03T16:16:40.7178695Z  fi 2024-04-03T16:16:40.7179024Z fi 2024-04-03T16:16:40.7179359Z  2024-04-03T16:16:40.7179800Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-04-03T16:16:40.7180429Z  # Clean up any old records first 2024-04-03T16:16:40.7181065Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-04-03T16:16:40.7181627Z fi 2024-04-03T16:16:40.7181966Z  2024-04-03T16:16:40.7182498Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-04-03T16:16:40.7183181Z cat /etc/hosts 2024-04-03T16:16:40.7191651Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:40.7192318Z env: 2024-04-03T16:16:40.7192669Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:40.7193098Z ##[endgroup] 2024-04-03T16:16:40.7219124Z + PT_DOMAIN=download.pytorch.org 2024-04-03T16:16:40.7225505Z ++ dig -4 +short download.pytorch.org 2024-04-03T16:16:40.7227330Z ++ tail -n1 2024-04-03T16:16:40.7316989Z + RESOLVED_IP=108.138.64.116 2024-04-03T16:16:40.7317672Z + '[' -z 108.138.64.116 ']' 2024-04-03T16:16:40.7318284Z + grep -r download.pytorch.org /etc/hosts 2024-04-03T16:16:40.7326851Z 18.160.41.23 download.pytorch.org 2024-04-03T16:16:40.7327998Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-04-03T16:16:40.7478398Z + echo '108.138.64.116 download.pytorch.org' 2024-04-03T16:16:40.7479886Z + sudo tee -a /etc/hosts 2024-04-03T16:16:40.7817199Z 108.138.64.116 download.pytorch.org 2024-04-03T16:16:40.7833949Z + cat /etc/hosts 2024-04-03T16:16:40.7841317Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-04-03T16:16:40.7856381Z ::1 localhost6 localhost6.localdomain6 2024-04-03T16:16:40.7857053Z 108.138.64.116 download.pytorch.org 2024-04-03T16:16:40.8051255Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-04-03T16:16:40.8052078Z with: 2024-04-03T16:16:40.8053279Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:40.8054681Z docker-build-dir: .ci/docker 2024-04-03T16:16:40.8055151Z working-directory: . 2024-04-03T16:16:40.8055716Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:40.8056387Z force-push: false 2024-04-03T16:16:40.8056756Z env: 2024-04-03T16:16:40.8057067Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:40.8057509Z ##[endgroup] 2024-04-03T16:16:40.8083543Z ##[group]Run set -ex 2024-04-03T16:16:40.8083990Z set -ex 2024-04-03T16:16:40.8084357Z  2024-04-03T16:16:40.8085039Z # If the docker build directory or the build script doesn't exist, the action will 2024-04-03T16:16:40.8086306Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-04-03T16:16:40.8087314Z # job could then download the pre-built image as usual 2024-04-03T16:16:40.8088250Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-04-03T16:16:40.8089100Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-04-03T16:16:40.8089899Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-04-03T16:16:40.8090582Z  2024-04-03T16:16:40.8091220Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-04-03T16:16:40.8092183Z  exit 0 2024-04-03T16:16:40.8092535Z else 2024-04-03T16:16:40.8092987Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-04-03T16:16:40.8093550Z fi 2024-04-03T16:16:40.8093872Z  2024-04-03T16:16:40.8094454Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-04-03T16:16:40.8095543Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-04-03T16:16:40.8096512Z  # use it as it is, but first let's extract the tag 2024-04-03T16:16:40.8097407Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-04-03T16:16:40.8098317Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-04-03T16:16:40.8099195Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-04-03T16:16:40.8099878Z else 2024-04-03T16:16:40.8100417Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-04-03T16:16:40.8101258Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-04-03T16:16:40.8102373Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-04-03T16:16:40.8103321Z fi 2024-04-03T16:16:40.8111471Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:40.8112095Z env: 2024-04-03T16:16:40.8112412Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:40.8112848Z REPO_NAME: pytorch 2024-04-03T16:16:40.8114109Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:40.8115455Z DOCKER_BUILD_DIR: .ci/docker 2024-04-03T16:16:40.8116076Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:40.8116723Z ##[endgroup] 2024-04-03T16:16:40.8142629Z + [[ ! -d .ci/docker ]] 2024-04-03T16:16:40.8143149Z + [[ ! -f .ci/docker/build.sh ]] 2024-04-03T16:16:40.8143633Z + echo skip=false 2024-04-03T16:16:40.8145622Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-04-03T16:16:40.8150305Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:40.8151589Z ++ awk -F '[:,]' '{print $2}' 2024-04-03T16:16:40.8161286Z + DOCKER_TAG=16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:40.8162127Z + echo docker-tag=16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:40.8164122Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:40.8198830Z ##[group]Run set +e 2024-04-03T16:16:40.8199750Z set +e 2024-04-03T16:16:40.8200145Z set -x 2024-04-03T16:16:40.8200481Z  2024-04-03T16:16:40.8200817Z login() { 2024-04-03T16:16:40.8201650Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-04-03T16:16:40.8202561Z } 2024-04-03T16:16:40.8202884Z  2024-04-03T16:16:40.8203265Z retry () { 2024-04-03T16:16:40.8203702Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-04-03T16:16:40.8204260Z } 2024-04-03T16:16:40.8204607Z  2024-04-03T16:16:40.8204962Z retry login "${DOCKER_REGISTRY}" 2024-04-03T16:16:40.8205475Z  2024-04-03T16:16:40.8206056Z # Check if image already exists, if it does then skip building it 2024-04-03T16:16:40.8206945Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-04-03T16:16:40.8207556Z  exit 0 2024-04-03T16:16:40.8207918Z fi 2024-04-03T16:16:40.8208257Z  2024-04-03T16:16:40.8209078Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-04-03T16:16:40.8210124Z # be empty. The default action would be to continue rebuild the image 2024-04-03T16:16:40.8211053Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-04-03T16:16:40.8211884Z  # if we're on the base branch then use the parent commit 2024-04-03T16:16:40.8212630Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-04-03T16:16:40.8213169Z else 2024-04-03T16:16:40.8213733Z  # otherwise we're on a PR, so use the most recent base commit 2024-04-03T16:16:40.8214595Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-04-03T16:16:40.8215227Z fi 2024-04-03T16:16:40.8215548Z  2024-04-03T16:16:40.8215929Z if [[ -z "${MERGE_BASE}" ]]; then 2024-04-03T16:16:40.8216553Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-04-03T16:16:40.8217115Z  2024-04-03T16:16:40.8217923Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-04-03T16:16:40.8218885Z  exit 0 2024-04-03T16:16:40.8219249Z fi 2024-04-03T16:16:40.8219565Z  2024-04-03T16:16:40.8220100Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-04-03T16:16:40.8221330Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-04-03T16:16:40.8222414Z  exit 1 2024-04-03T16:16:40.8259003Z fi 2024-04-03T16:16:40.8259374Z  2024-04-03T16:16:40.8260038Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-04-03T16:16:40.8261339Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-04-03T16:16:40.8262526Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-04-03T16:16:40.8263838Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-04-03T16:16:40.8265331Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-04-03T16:16:40.8266201Z fi 2024-04-03T16:16:40.8266535Z  2024-04-03T16:16:40.8267204Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-04-03T16:16:40.8275678Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:40.8276332Z env: 2024-04-03T16:16:40.8276672Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:40.8277147Z DOCKER_BUILD_DIR: .ci/docker 2024-04-03T16:16:40.8277759Z BASE_REVISION: 1d52c2d98507759a2143d52b922c541dbe36b6bf 2024-04-03T16:16:40.8279768Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:40.8281185Z DOCKER_TAG: 16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:40.8281944Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:40.8282600Z ##[endgroup] 2024-04-03T16:16:40.8307879Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:40.8308588Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:40.8309810Z + aws ecr get-login-password --region us-east-1 2024-04-03T16:16:40.8311255Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:41.3419530Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-04-03T16:16:41.3420856Z Configure a credential helper to remove this warning. See 2024-04-03T16:16:41.3422069Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-04-03T16:16:41.3422745Z 2024-04-03T16:16:41.3422921Z Login Succeeded 2024-04-03T16:16:41.3435534Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:41.5564975Z { 2024-04-03T16:16:41.5565887Z "schemaVersion": 2, 2024-04-03T16:16:41.5566963Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-04-03T16:16:41.5567740Z "config": { 2024-04-03T16:16:41.5568317Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-04-03T16:16:41.5569064Z "size": 45677, 2024-04-03T16:16:41.5570207Z "digest": "sha256:4f8185cfd17efcb172142a7ee4ddc70e9bf6e340c2b0e42aa0def9ad36662df7" 2024-04-03T16:16:41.5571716Z }, 2024-04-03T16:16:41.5572332Z "layers": [ 2024-04-03T16:16:41.5572915Z { 2024-04-03T16:16:41.5574077Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5574844Z "size": 28580681, 2024-04-03T16:16:41.5575569Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-04-03T16:16:41.5576343Z }, 2024-04-03T16:16:41.5576622Z { 2024-04-03T16:16:41.5577221Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5578695Z "size": 7943451, 2024-04-03T16:16:41.5579616Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-04-03T16:16:41.5580391Z }, 2024-04-03T16:16:41.5580686Z { 2024-04-03T16:16:41.5581213Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5581917Z "size": 55728572, 2024-04-03T16:16:41.5582605Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-04-03T16:16:41.5583349Z }, 2024-04-03T16:16:41.5583642Z { 2024-04-03T16:16:41.5584179Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5585326Z "size": 186, 2024-04-03T16:16:41.5586023Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-04-03T16:16:41.5586810Z }, 2024-04-03T16:16:41.5587094Z { 2024-04-03T16:16:41.5587624Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5588330Z "size": 6886, 2024-04-03T16:16:41.5588765Z + exit 0 2024-04-03T16:16:41.5589392Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-04-03T16:16:41.5590482Z }, 2024-04-03T16:16:41.5591052Z { 2024-04-03T16:16:41.5592502Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5594026Z "size": 1329001756, 2024-04-03T16:16:41.5595571Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-04-03T16:16:41.5597274Z }, 2024-04-03T16:16:41.5597923Z { 2024-04-03T16:16:41.5599375Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5600855Z "size": 62501, 2024-04-03T16:16:41.5602115Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-04-03T16:16:41.5603540Z }, 2024-04-03T16:16:41.5604045Z { 2024-04-03T16:16:41.5605241Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5605971Z "size": 1684, 2024-04-03T16:16:41.5606637Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-04-03T16:16:41.5607381Z }, 2024-04-03T16:16:41.5607674Z { 2024-04-03T16:16:41.5608212Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5608896Z "size": 1523, 2024-04-03T16:16:41.5609594Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-04-03T16:16:41.5610389Z }, 2024-04-03T16:16:41.5610667Z { 2024-04-03T16:16:41.5611201Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5611903Z "size": 2528295403, 2024-04-03T16:16:41.5612591Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-04-03T16:16:41.5613366Z }, 2024-04-03T16:16:41.5613663Z { 2024-04-03T16:16:41.5614182Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5615081Z "size": 86016, 2024-04-03T16:16:41.5615772Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-04-03T16:16:41.5616532Z }, 2024-04-03T16:16:41.5616826Z { 2024-04-03T16:16:41.5617360Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5618037Z "size": 1827, 2024-04-03T16:16:41.5618708Z "digest": "sha256:3704f77640802bd5e9366068958aeb7586ebecf59ebdc0456e254c95e334c612" 2024-04-03T16:16:41.5619468Z }, 2024-04-03T16:16:41.5619745Z { 2024-04-03T16:16:41.5620281Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5620981Z "size": 246729069, 2024-04-03T16:16:41.5621687Z "digest": "sha256:a24efd49f0c8dd1cb4bc8c5f3dd7f7be33b180bc006ec784e60a63c1d993c8b2" 2024-04-03T16:16:41.5622472Z }, 2024-04-03T16:16:41.5622769Z { 2024-04-03T16:16:41.5623290Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5623986Z "size": 544, 2024-04-03T16:16:41.5624679Z "digest": "sha256:7d4d2c342f82b5da5de28dc26020e6cb2ceab9d2610023b0c13a38c10bc9ab21" 2024-04-03T16:16:41.5625581Z }, 2024-04-03T16:16:41.5625881Z { 2024-04-03T16:16:41.5626416Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5627092Z "size": 1281, 2024-04-03T16:16:41.5627778Z "digest": "sha256:6a9e70a8826103fe3e60bd5a423d969cf2f0efaba67b888866cb105a7f76c656" 2024-04-03T16:16:41.5628559Z }, 2024-04-03T16:16:41.5628839Z { 2024-04-03T16:16:41.5629372Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5630064Z "size": 483, 2024-04-03T16:16:41.5630720Z "digest": "sha256:1173f47dcf05d9c022a4e47b07f18510823ac64bc2afc714fe72a75d9360cfe8" 2024-04-03T16:16:41.5631490Z }, 2024-04-03T16:16:41.5631787Z { 2024-04-03T16:16:41.5632307Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5632998Z "size": 91704809, 2024-04-03T16:16:41.5633705Z "digest": "sha256:325d4583fff5a86485e69ad6aa7002e2bc626ed10e2c7cd9ac669e3d7df1de9a" 2024-04-03T16:16:41.5634473Z }, 2024-04-03T16:16:41.5634770Z { 2024-04-03T16:16:41.5635305Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5635980Z "size": 3256, 2024-04-03T16:16:41.5636796Z "digest": "sha256:c0d6db5efa7d7e4e7dc568ec1b7c37323594066efb8a43e02f92609a089e9415" 2024-04-03T16:16:41.5637602Z }, 2024-04-03T16:16:41.5637881Z { 2024-04-03T16:16:41.5638418Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5639397Z "size": 1904, 2024-04-03T16:16:41.5640176Z "digest": "sha256:75830f3a12e2b08a1bce7cf82deb21eaa4f605ea0dec0f1be438ca06e2484710" 2024-04-03T16:16:41.5640965Z }, 2024-04-03T16:16:41.5641261Z { 2024-04-03T16:16:41.5641784Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5642477Z "size": 699, 2024-04-03T16:16:41.5643136Z "digest": "sha256:e857029fb4018352e1abb6c30f3e35072be04e7a6856b20466383d299aa4761c" 2024-04-03T16:16:41.5643894Z }, 2024-04-03T16:16:41.5644192Z { 2024-04-03T16:16:41.5644726Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5645408Z "size": 2776066004, 2024-04-03T16:16:41.5646100Z "digest": "sha256:9fc6458e826319cca72932b0c615287b9d5f01a3b89652af71488308a4f06a92" 2024-04-03T16:16:41.5646863Z }, 2024-04-03T16:16:41.5647141Z { 2024-04-03T16:16:41.5647678Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5648373Z "size": 379, 2024-04-03T16:16:41.5649037Z "digest": "sha256:3bcad833b4fc2f8160bb638efd23df97e3e454d84ad113315f81d827a4d17776" 2024-04-03T16:16:41.5649823Z }, 2024-04-03T16:16:41.5650119Z { 2024-04-03T16:16:41.5650638Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5651333Z "size": 11870, 2024-04-03T16:16:41.5652003Z "digest": "sha256:11bbf370b108d68839d9342b11b6ae8074ac90c76121e76a4917af2eced7b5f1" 2024-04-03T16:16:41.5652946Z }, 2024-04-03T16:16:41.5653245Z { 2024-04-03T16:16:41.5653762Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5654452Z "size": 802, 2024-04-03T16:16:41.5655116Z "digest": "sha256:e7825e6a30f3c88c4f86fd89d4f348976237e8f0d0f45434d803e80463ff0d05" 2024-04-03T16:16:41.5655862Z }, 2024-04-03T16:16:41.5656157Z { 2024-04-03T16:16:41.5656702Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5657380Z "size": 106, 2024-04-03T16:16:41.5658086Z "digest": "sha256:ea7ce2e8ea6d2e9902ffdda8bec1db58a85eb6d4ddb722f8a90b37eb15c7a10c" 2024-04-03T16:16:41.5658892Z }, 2024-04-03T16:16:41.5659170Z { 2024-04-03T16:16:41.5659702Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5660399Z "size": 503, 2024-04-03T16:16:41.5661046Z "digest": "sha256:73368bd9e761796f178a165c16b753ed00bcfe2ec9cb13f384444ff8766ec802" 2024-04-03T16:16:41.5661809Z }, 2024-04-03T16:16:41.5662102Z { 2024-04-03T16:16:41.5662629Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5663322Z "size": 121477165, 2024-04-03T16:16:41.5664025Z "digest": "sha256:c7972c3f0de1bedf9554a091b96ed330785777db15c93df2a74191903b42335c" 2024-04-03T16:16:41.5664783Z }, 2024-04-03T16:16:41.5665078Z { 2024-04-03T16:16:41.5665619Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5666298Z "size": 109, 2024-04-03T16:16:41.5666972Z "digest": "sha256:3261cbfdbc87f311a080002dcb669c3641a7a9f4b6138855391fa9e8857be866" 2024-04-03T16:16:41.5667739Z }, 2024-04-03T16:16:41.5668015Z { 2024-04-03T16:16:41.5668548Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5669290Z "size": 556, 2024-04-03T16:16:41.5669942Z "digest": "sha256:bbb4242b1c5a4607293ed61a1ea775d5c98126930c02622f1a39239de458c66a" 2024-04-03T16:16:41.5670710Z }, 2024-04-03T16:16:41.5671008Z { 2024-04-03T16:16:41.5671527Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5672235Z "size": 1175573, 2024-04-03T16:16:41.5672916Z "digest": "sha256:6e5a2ed1895563c32a245908e9e501fd10c39a0cf82be2b79895a9ec5e40cf47" 2024-04-03T16:16:41.5673666Z }, 2024-04-03T16:16:41.5673960Z { 2024-04-03T16:16:41.5674651Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5675348Z "size": 103, 2024-04-03T16:16:41.5676023Z "digest": "sha256:c40fcbdc71225ac252b93ceef78a66e28442a09307e07d254554b531b78c22ce" 2024-04-03T16:16:41.5676803Z }, 2024-04-03T16:16:41.5677081Z { 2024-04-03T16:16:41.5677620Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5678316Z "size": 1489, 2024-04-03T16:16:41.5678977Z "digest": "sha256:d955fef15356d1bb6a7df089ee9e3518db48a506787838a339ece35367fe586c" 2024-04-03T16:16:41.5680130Z }, 2024-04-03T16:16:41.5680432Z { 2024-04-03T16:16:41.5680958Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5681675Z "size": 442746057, 2024-04-03T16:16:41.5682398Z "digest": "sha256:2de9ee0e31aafbc4cec6204b3f84662b32bbebfaf36c499c94d55a30a9a8f11f" 2024-04-03T16:16:41.5683172Z }, 2024-04-03T16:16:41.5683468Z { 2024-04-03T16:16:41.5684005Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5684693Z "size": 160, 2024-04-03T16:16:41.5685386Z "digest": "sha256:91caa7e72a27c8440e52efb42e0f679e3fae0cc1f5dd830f1caeadceb2b83862" 2024-04-03T16:16:41.5686182Z }, 2024-04-03T16:16:41.5686460Z { 2024-04-03T16:16:41.5686996Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5687689Z "size": 562, 2024-04-03T16:16:41.5688351Z "digest": "sha256:0d897f9ab51125cce55bb448ac4baecf615a48b655f0a60e1623b84c96d8b1e4" 2024-04-03T16:16:41.5689132Z }, 2024-04-03T16:16:41.5689426Z { 2024-04-03T16:16:41.5689948Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5690779Z "size": 33264440, 2024-04-03T16:16:41.5691468Z "digest": "sha256:76a6353e12ba5344141db8704c2f33ac235c09d8d66e5bba54bf66fd6bcba47f" 2024-04-03T16:16:41.5692218Z }, 2024-04-03T16:16:41.5692513Z { 2024-04-03T16:16:41.5693048Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5693723Z "size": 104, 2024-04-03T16:16:41.5694396Z "digest": "sha256:d27855b0149450966d7e8b078ede420de447c44ea5c39dbdf25be6e5637a9a21" 2024-04-03T16:16:41.5695168Z }, 2024-04-03T16:16:41.5695448Z { 2024-04-03T16:16:41.5695983Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5696676Z "size": 425, 2024-04-03T16:16:41.5697326Z "digest": "sha256:04a1e7a3094076d0fcb62a8b1aa474b68435c62a146ae806fc8274ba15fef0f4" 2024-04-03T16:16:41.5698095Z }, 2024-04-03T16:16:41.5698388Z { 2024-04-03T16:16:41.5698906Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5699606Z "size": 20262043, 2024-04-03T16:16:41.5700297Z "digest": "sha256:2d6278964762a93ae206b2831482b4bf8811b33357d8164f34f9db7cb081b31a" 2024-04-03T16:16:41.5701044Z }, 2024-04-03T16:16:41.5701323Z { 2024-04-03T16:16:41.5701854Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5702530Z "size": 437, 2024-04-03T16:16:41.5703212Z "digest": "sha256:87347beefd8eb2bc14f7aaa327448c54711c5019ca19ec9f1662329a758fd861" 2024-04-03T16:16:41.5703989Z }, 2024-04-03T16:16:41.5704264Z { 2024-04-03T16:16:41.5704802Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5705494Z "size": 699, 2024-04-03T16:16:41.5706139Z "digest": "sha256:e857029fb4018352e1abb6c30f3e35072be04e7a6856b20466383d299aa4761c" 2024-04-03T16:16:41.5706899Z }, 2024-04-03T16:16:41.5707196Z { 2024-04-03T16:16:41.5707720Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5708414Z "size": 142, 2024-04-03T16:16:41.5709107Z "digest": "sha256:4fadf75753a0920a54bf82c91b84dadb65cfffff7fd21aaba72289be2fc972ac" 2024-04-03T16:16:41.5709906Z }, 2024-04-03T16:16:41.5710182Z { 2024-04-03T16:16:41.5710718Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5711410Z "size": 136, 2024-04-03T16:16:41.5712172Z "digest": "sha256:695c95e1ff7f1c1e44815252318cd9a4d2ae8872251afa6af8b922dfe1792fc9" 2024-04-03T16:16:41.5712960Z }, 2024-04-03T16:16:41.5713255Z { 2024-04-03T16:16:41.5713774Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5714463Z "size": 32, 2024-04-03T16:16:41.5715137Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-04-03T16:16:41.5715902Z }, 2024-04-03T16:16:41.5716195Z { 2024-04-03T16:16:41.5716727Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5717403Z "size": 190, 2024-04-03T16:16:41.5718073Z "digest": "sha256:0c21214a7be681addf0f5b401166f05f3e7c3030d312e7d0435e063f1cfe3437" 2024-04-03T16:16:41.5718855Z }, 2024-04-03T16:16:41.5719419Z { 2024-04-03T16:16:41.5720101Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5720797Z "size": 562, 2024-04-03T16:16:41.5721460Z "digest": "sha256:bde926021ef3b3cdd81a352c85a50992d61b85403bd4bf0136b69eface867cc4" 2024-04-03T16:16:41.5722238Z }, 2024-04-03T16:16:41.5722542Z { 2024-04-03T16:16:41.5723062Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5723754Z "size": 43165786, 2024-04-03T16:16:41.5724444Z "digest": "sha256:fa513a4972bd3d903ae1d31755841a38f6c9fbb4a9590423197f988e2111e12a" 2024-04-03T16:16:41.5725195Z }, 2024-04-03T16:16:41.5725490Z { 2024-04-03T16:16:41.5726024Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5726706Z "size": 106, 2024-04-03T16:16:41.5727385Z "digest": "sha256:fbb14e703cd0a28afa275d4af5cc894c36610056ff58b28211e73c12228a4195" 2024-04-03T16:16:41.5728162Z }, 2024-04-03T16:16:41.5728608Z { 2024-04-03T16:16:41.5729147Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5729835Z "size": 1192, 2024-04-03T16:16:41.5730513Z "digest": "sha256:2f633f5b71fab54bc7e9b363dcb1dafe3ce70ba7a78f5409a13b10f04ecd57e4" 2024-04-03T16:16:41.5731296Z }, 2024-04-03T16:16:41.5731588Z { 2024-04-03T16:16:41.5732115Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5732816Z "size": 699, 2024-04-03T16:16:41.5733476Z "digest": "sha256:e857029fb4018352e1abb6c30f3e35072be04e7a6856b20466383d299aa4761c" 2024-04-03T16:16:41.5734221Z }, 2024-04-03T16:16:41.5734514Z { 2024-04-03T16:16:41.5735052Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5735724Z "size": 137, 2024-04-03T16:16:41.5736380Z "digest": "sha256:939366b420fcf364a880214ffd86f8d4437f1865c82da7e835260996177ff4bc" 2024-04-03T16:16:41.5737132Z }, 2024-04-03T16:16:41.5737408Z { 2024-04-03T16:16:41.5737949Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5738640Z "size": 120, 2024-04-03T16:16:41.5739293Z "digest": "sha256:aac618ecc588e8b596117028b6295a96437d1adb6751cc751ba89528b542a885" 2024-04-03T16:16:41.5740061Z }, 2024-04-03T16:16:41.5740354Z { 2024-04-03T16:16:41.5740879Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5741579Z "size": 1704563257, 2024-04-03T16:16:41.5742271Z "digest": "sha256:f884b36705755d621bb47cf7d82d1ef640041c2b66a5b534777d477c3b284182" 2024-04-03T16:16:41.5743006Z }, 2024-04-03T16:16:41.5743302Z { 2024-04-03T16:16:41.5743834Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5744511Z "size": 170, 2024-04-03T16:16:41.5745184Z "digest": "sha256:97cd1963553999c53b20fc9ffd6a2a068140cfb94eff5b30aa334af259eba2bd" 2024-04-03T16:16:41.5745954Z }, 2024-04-03T16:16:41.5746232Z { 2024-04-03T16:16:41.5746765Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5747468Z "size": 1840, 2024-04-03T16:16:41.5748123Z "digest": "sha256:189619a5dde8c2add80a644743b37d1e285b62892f48e536d1826b029e7da612" 2024-04-03T16:16:41.5748886Z }, 2024-04-03T16:16:41.5749179Z { 2024-04-03T16:16:41.5749813Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5750533Z "size": 7529791, 2024-04-03T16:16:41.5751246Z "digest": "sha256:1ffff58a595ca2f43c30ba4ca5fea2aaa53fbe2e6484b6b616c7b0da42f28d4f" 2024-04-03T16:16:41.5752028Z }, 2024-04-03T16:16:41.5752321Z { 2024-04-03T16:16:41.5752858Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5753536Z "size": 164, 2024-04-03T16:16:41.5754217Z "digest": "sha256:bf176a055b2c55096ef30ca6e2690e06cd247f0bc8b7d8afebc3d1174756c5d0" 2024-04-03T16:16:41.5754988Z }, 2024-04-03T16:16:41.5755266Z { 2024-04-03T16:16:41.5755796Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5756494Z "size": 7938, 2024-04-03T16:16:41.5757141Z "digest": "sha256:4fe2333aa4a116952b95d595191ffde94190713c35f27f8f88e23c618cc45217" 2024-04-03T16:16:41.5757903Z }, 2024-04-03T16:16:41.5758193Z { 2024-04-03T16:16:41.5758711Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5759605Z "size": 8069, 2024-04-03T16:16:41.5760296Z "digest": "sha256:d40af861237a8e36cd2b3c8079cd21b9d10cab05f0a6c88dff474819ea529d35" 2024-04-03T16:16:41.5761060Z }, 2024-04-03T16:16:41.5761354Z { 2024-04-03T16:16:41.5761886Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5762561Z "size": 304, 2024-04-03T16:16:41.5763221Z "digest": "sha256:a8168f6485116ca2cd40d8246452df93c4164e01e2045d5f3481d52eac976170" 2024-04-03T16:16:41.5763978Z }, 2024-04-03T16:16:41.5764255Z { 2024-04-03T16:16:41.5764788Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5765483Z "size": 7620311, 2024-04-03T16:16:41.5766285Z "digest": "sha256:79c82d9080d8a32606b02dc452f2af15d6f5d6dd74858968b11f5da1fe2c7c98" 2024-04-03T16:16:41.5767056Z }, 2024-04-03T16:16:41.5767348Z { 2024-04-03T16:16:41.5767864Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5768555Z "size": 108, 2024-04-03T16:16:41.5769275Z "digest": "sha256:48bd505552555bd866a4dc0460e69df90b0a7594bdd0c880d0a13b1af200797c" 2024-04-03T16:16:41.5770028Z }, 2024-04-03T16:16:41.5770323Z { 2024-04-03T16:16:41.5770858Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5771538Z "size": 54145489, 2024-04-03T16:16:41.5772238Z "digest": "sha256:69bf265ec18d4e6ae8b71cadc9221d44a08818694e42d8f42d408f9aca710c42" 2024-04-03T16:16:41.5773014Z }, 2024-04-03T16:16:41.5773290Z { 2024-04-03T16:16:41.5773823Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5774517Z "size": 499, 2024-04-03T16:16:41.5775196Z "digest": "sha256:076ea06db0f9eee3b5a061835ac2d815acc050abdfb959f7243cebda5f665261" 2024-04-03T16:16:41.5775978Z }, 2024-04-03T16:16:41.5776278Z { 2024-04-03T16:16:41.5776794Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5777488Z "size": 1496846361, 2024-04-03T16:16:41.5778210Z "digest": "sha256:acc43aa33d6b69e371d50f72c729102a8a5b4b2dc9a3b47c60587ef83dbf6e6e" 2024-04-03T16:16:41.5778972Z }, 2024-04-03T16:16:41.5779267Z { 2024-04-03T16:16:41.5779800Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5780468Z "size": 106, 2024-04-03T16:16:41.5781148Z "digest": "sha256:fba378c7a8933d3debafa37c2017a761e7472b20cdb9329b1bd6163955dba45f" 2024-04-03T16:16:41.5781925Z }, 2024-04-03T16:16:41.5782203Z { 2024-04-03T16:16:41.5782734Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5783425Z "size": 445, 2024-04-03T16:16:41.5784077Z "digest": "sha256:7715b3f284b54b2ed0a173c834704eb0415c2d712fe25dabda097bca80ba8e0a" 2024-04-03T16:16:41.5784850Z }, 2024-04-03T16:16:41.5785141Z { 2024-04-03T16:16:41.5785659Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5786350Z "size": 46248381, 2024-04-03T16:16:41.5787153Z "digest": "sha256:1f393e236d256741df5a8775660b74f734fb7c21a8e7cc7d7beda9a359d2a6c4" 2024-04-03T16:16:41.5787918Z }, 2024-04-03T16:16:41.5788213Z { 2024-04-03T16:16:41.5788750Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5789423Z "size": 111, 2024-04-03T16:16:41.5790086Z "digest": "sha256:60489ca1c1f78f1a38c5a5e7a25003821303e2515576e1f3c3d576f27090ae8e" 2024-04-03T16:16:41.5790839Z }, 2024-04-03T16:16:41.5791116Z { 2024-04-03T16:16:41.5791650Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5792344Z "size": 32, 2024-04-03T16:16:41.5793001Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-04-03T16:16:41.5793788Z }, 2024-04-03T16:16:41.5794084Z { 2024-04-03T16:16:41.5794601Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5795295Z "size": 32, 2024-04-03T16:16:41.5795972Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-04-03T16:16:41.5796735Z }, 2024-04-03T16:16:41.5797036Z { 2024-04-03T16:16:41.5797570Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-04-03T16:16:41.5798244Z "size": 32, 2024-04-03T16:16:41.5798924Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-04-03T16:16:41.5799883Z } 2024-04-03T16:16:41.5800163Z ] 2024-04-03T16:16:41.5800460Z } 2024-04-03T16:16:41.5939691Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-04-03T16:16:41.5940337Z tag=${ECR_DOCKER_IMAGE##*/} 2024-04-03T16:16:41.5941019Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-04-03T16:16:41.5949952Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:41.5950717Z env: 2024-04-03T16:16:41.5951049Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:41.5952332Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:41.5953667Z ##[endgroup] 2024-04-03T16:16:41.5983372Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9-16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:41.6048649Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-04-03T16:16:41.6049392Z with: 2024-04-03T16:16:41.6050565Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:41.6052061Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:41.6052680Z env: 2024-04-03T16:16:41.6053011Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:41.6053450Z ##[endgroup] 2024-04-03T16:16:41.6076403Z ##[group]Run set -x 2024-04-03T16:16:41.6076820Z set -x 2024-04-03T16:16:41.6077189Z set +e 2024-04-03T16:16:41.6077544Z  2024-04-03T16:16:41.6077861Z login() { 2024-04-03T16:16:41.6078689Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-04-03T16:16:41.6079854Z } 2024-04-03T16:16:41.6080173Z  2024-04-03T16:16:41.6080566Z retry () { 2024-04-03T16:16:41.6081033Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-04-03T16:16:41.6081586Z } 2024-04-03T16:16:41.6081918Z  2024-04-03T16:16:41.6082282Z retry login "${DOCKER_REGISTRY}" 2024-04-03T16:16:41.6082796Z  2024-04-03T16:16:41.6083126Z set -e 2024-04-03T16:16:41.6083694Z # ignore output since only exit code is used for conditional 2024-04-03T16:16:41.6084577Z # only pull docker image if it's not available locally 2024-04-03T16:16:41.6085552Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-04-03T16:16:41.6086422Z  retry docker pull "${DOCKER_IMAGE}" 2024-04-03T16:16:41.6086947Z fi 2024-04-03T16:16:41.6094608Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:16:41.6095224Z env: 2024-04-03T16:16:41.6095556Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:16:41.6096809Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:41.6098292Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:41.6098938Z ##[endgroup] 2024-04-03T16:16:41.6123559Z + set +e 2024-04-03T16:16:41.6124181Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:41.6124980Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:41.6127255Z + aws ecr get-login-password --region us-east-1 2024-04-03T16:16:41.6128682Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-04-03T16:16:42.1150338Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-04-03T16:16:42.1151534Z Configure a credential helper to remove this warning. See 2024-04-03T16:16:42.1153004Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-04-03T16:16:42.1153701Z 2024-04-03T16:16:42.1153891Z Login Succeeded 2024-04-03T16:16:42.1166011Z + set -e 2024-04-03T16:16:42.1167432Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:42.1380975Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:42.1383387Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:16:42.3520583Z 16884143090f148aa56809ceba26eeb699b5dc49: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9 2024-04-03T16:16:42.3522254Z 7a2c55901189: Pulling fs layer 2024-04-03T16:16:42.3523270Z 224fe954d725: Pulling fs layer 2024-04-03T16:16:42.3524282Z 75722010b82e: Pulling fs layer 2024-04-03T16:16:42.3525044Z d527cbbb87e3: Pulling fs layer 2024-04-03T16:16:42.3526472Z b57676e46aee: Pulling fs layer 2024-04-03T16:16:42.3527530Z a8c1e85b5e14: Pulling fs layer 2024-04-03T16:16:42.3528476Z a41a8d1c11c8: Pulling fs layer 2024-04-03T16:16:42.3529400Z 0c1227890755: Pulling fs layer 2024-04-03T16:16:42.3530339Z d8d1234baab3: Pulling fs layer 2024-04-03T16:16:42.3531270Z 7ed32bc8e469: Pulling fs layer 2024-04-03T16:16:42.3532227Z ec1e7978c1fe: Pulling fs layer 2024-04-03T16:16:42.3533260Z 3704f7764080: Pulling fs layer 2024-04-03T16:16:42.3534194Z a24efd49f0c8: Pulling fs layer 2024-04-03T16:16:42.3535141Z 7d4d2c342f82: Pulling fs layer 2024-04-03T16:16:42.3536111Z 6a9e70a88261: Pulling fs layer 2024-04-03T16:16:42.3537069Z 1173f47dcf05: Pulling fs layer 2024-04-03T16:16:42.3537812Z a8c1e85b5e14: Waiting 2024-04-03T16:16:42.3538229Z 325d4583fff5: Pulling fs layer 2024-04-03T16:16:42.3538698Z c0d6db5efa7d: Pulling fs layer 2024-04-03T16:16:42.3539130Z a41a8d1c11c8: Waiting 2024-04-03T16:16:42.3539513Z 0c1227890755: Waiting 2024-04-03T16:16:42.3539928Z 75830f3a12e2: Pulling fs layer 2024-04-03T16:16:42.3540359Z d8d1234baab3: Waiting 2024-04-03T16:16:42.3540751Z 7ed32bc8e469: Waiting 2024-04-03T16:16:42.3541139Z d527cbbb87e3: Waiting 2024-04-03T16:16:42.3541528Z e857029fb401: Pulling fs layer 2024-04-03T16:16:42.3541972Z ec1e7978c1fe: Waiting 2024-04-03T16:16:42.3542363Z a24efd49f0c8: Waiting 2024-04-03T16:16:42.3542732Z 7d4d2c342f82: Waiting 2024-04-03T16:16:42.3543134Z 9fc6458e8263: Pulling fs layer 2024-04-03T16:16:42.3543573Z 6a9e70a88261: Waiting 2024-04-03T16:16:42.3543950Z 3704f7764080: Waiting 2024-04-03T16:16:42.3544353Z 3bcad833b4fc: Pulling fs layer 2024-04-03T16:16:42.3544821Z 11bbf370b108: Pulling fs layer 2024-04-03T16:16:42.3545249Z 325d4583fff5: Waiting 2024-04-03T16:16:42.3545633Z 1173f47dcf05: Waiting 2024-04-03T16:16:42.3546035Z e7825e6a30f3: Pulling fs layer 2024-04-03T16:16:42.3546463Z e857029fb401: Waiting 2024-04-03T16:16:42.3546846Z 9fc6458e8263: Waiting 2024-04-03T16:16:42.3547210Z c0d6db5efa7d: Waiting 2024-04-03T16:16:42.3547629Z ea7ce2e8ea6d: Pulling fs layer 2024-04-03T16:16:42.3548103Z 73368bd9e761: Pulling fs layer 2024-04-03T16:16:42.3548530Z b57676e46aee: Waiting 2024-04-03T16:16:42.3548916Z 3bcad833b4fc: Waiting 2024-04-03T16:16:42.3549302Z e7825e6a30f3: Waiting 2024-04-03T16:16:42.3549677Z ea7ce2e8ea6d: Waiting 2024-04-03T16:16:42.3550087Z c7972c3f0de1: Pulling fs layer 2024-04-03T16:16:42.3550530Z 75830f3a12e2: Waiting 2024-04-03T16:16:42.3550914Z 3261cbfdbc87: Pulling fs layer 2024-04-03T16:16:42.3551359Z c7972c3f0de1: Waiting 2024-04-03T16:16:42.3551779Z bbb4242b1c5a: Pulling fs layer 2024-04-03T16:16:42.3552208Z 73368bd9e761: Waiting 2024-04-03T16:16:42.3552608Z 6e5a2ed18955: Pulling fs layer 2024-04-03T16:16:42.3553051Z bbb4242b1c5a: Waiting 2024-04-03T16:16:42.3553420Z 11bbf370b108: Waiting 2024-04-03T16:16:42.3553824Z c40fcbdc7122: Pulling fs layer 2024-04-03T16:16:42.3554292Z d955fef15356: Pulling fs layer 2024-04-03T16:16:42.3554741Z 2de9ee0e31aa: Pulling fs layer 2024-04-03T16:16:42.3555207Z 91caa7e72a27: Pulling fs layer 2024-04-03T16:16:42.3555664Z 6e5a2ed18955: Waiting 2024-04-03T16:16:42.3556034Z d955fef15356: Waiting 2024-04-03T16:16:42.3556438Z 0d897f9ab511: Pulling fs layer 2024-04-03T16:16:42.3556886Z 2de9ee0e31aa: Waiting 2024-04-03T16:16:42.3557259Z 91caa7e72a27: Waiting 2024-04-03T16:16:42.3557661Z 76a6353e12ba: Pulling fs layer 2024-04-03T16:16:42.3558102Z 0d897f9ab511: Waiting 2024-04-03T16:16:42.3558471Z c40fcbdc7122: Waiting 2024-04-03T16:16:42.3558870Z d27855b01494: Pulling fs layer 2024-04-03T16:16:42.3559531Z 76a6353e12ba: Waiting 2024-04-03T16:16:42.3560128Z 04a1e7a30940: Pulling fs layer 2024-04-03T16:16:42.3560564Z d27855b01494: Waiting 2024-04-03T16:16:42.3560962Z 2d6278964762: Pulling fs layer 2024-04-03T16:16:42.3561405Z 87347beefd8e: Pulling fs layer 2024-04-03T16:16:42.3561866Z 4fadf75753a0: Pulling fs layer 2024-04-03T16:16:42.3562329Z 695c95e1ff7f: Pulling fs layer 2024-04-03T16:16:42.3562773Z 4f4fb700ef54: Pulling fs layer 2024-04-03T16:16:42.3563234Z 0c21214a7be6: Pulling fs layer 2024-04-03T16:16:42.3563838Z bde926021ef3: Pulling fs layer 2024-04-03T16:16:42.3564285Z 04a1e7a30940: Waiting 2024-04-03T16:16:42.3564689Z fa513a4972bd: Pulling fs layer 2024-04-03T16:16:42.3565134Z 695c95e1ff7f: Waiting 2024-04-03T16:16:42.3565499Z 2d6278964762: Waiting 2024-04-03T16:16:42.3565879Z 87347beefd8e: Waiting 2024-04-03T16:16:42.3566287Z fbb14e703cd0: Pulling fs layer 2024-04-03T16:16:42.3566721Z bde926021ef3: Waiting 2024-04-03T16:16:42.3567109Z 4fadf75753a0: Waiting 2024-04-03T16:16:42.3567493Z fa513a4972bd: Waiting 2024-04-03T16:16:42.3567875Z 4f4fb700ef54: Waiting 2024-04-03T16:16:42.3568279Z 2f633f5b71fa: Pulling fs layer 2024-04-03T16:16:42.3568766Z 0c21214a7be6: Waiting 2024-04-03T16:16:42.3569138Z 2f633f5b71fa: Waiting 2024-04-03T16:16:42.3569522Z fbb14e703cd0: Waiting 2024-04-03T16:16:42.3569928Z 939366b420fc: Pulling fs layer 2024-04-03T16:16:42.3570375Z aac618ecc588: Pulling fs layer 2024-04-03T16:16:42.3570837Z f884b3670575: Pulling fs layer 2024-04-03T16:16:42.3571297Z 97cd19635539: Pulling fs layer 2024-04-03T16:16:42.3571750Z 189619a5dde8: Pulling fs layer 2024-04-03T16:16:42.3572223Z 1ffff58a595c: Pulling fs layer 2024-04-03T16:16:42.3572668Z aac618ecc588: Waiting 2024-04-03T16:16:42.3573036Z 939366b420fc: Waiting 2024-04-03T16:16:42.3573441Z bf176a055b2c: Pulling fs layer 2024-04-03T16:16:42.3573908Z f884b3670575: Waiting 2024-04-03T16:16:42.3574297Z 4fe2333aa4a1: Pulling fs layer 2024-04-03T16:16:42.3574745Z 97cd19635539: Waiting 2024-04-03T16:16:42.3575146Z d40af861237a: Pulling fs layer 2024-04-03T16:16:42.3575602Z a8168f648511: Pulling fs layer 2024-04-03T16:16:42.3576042Z bf176a055b2c: Waiting 2024-04-03T16:16:42.3576426Z 1ffff58a595c: Waiting 2024-04-03T16:16:42.3576799Z 4fe2333aa4a1: Waiting 2024-04-03T16:16:42.3577202Z 79c82d9080d8: Pulling fs layer 2024-04-03T16:16:42.3577644Z d40af861237a: Waiting 2024-04-03T16:16:42.3578027Z 48bd50555255: Pulling fs layer 2024-04-03T16:16:42.3578490Z 69bf265ec18d: Pulling fs layer 2024-04-03T16:16:42.3578928Z a8168f648511: Waiting 2024-04-03T16:16:42.3579321Z 076ea06db0f9: Pulling fs layer 2024-04-03T16:16:42.3579791Z acc43aa33d6b: Pulling fs layer 2024-04-03T16:16:42.3580236Z 48bd50555255: Waiting 2024-04-03T16:16:42.3580599Z 69bf265ec18d: Waiting 2024-04-03T16:16:42.3580980Z 79c82d9080d8: Waiting 2024-04-03T16:16:42.3581386Z fba378c7a893: Pulling fs layer 2024-04-03T16:16:42.3581815Z 076ea06db0f9: Waiting 2024-04-03T16:16:42.3582224Z 7715b3f284b5: Pulling fs layer 2024-04-03T16:16:42.3582672Z acc43aa33d6b: Waiting 2024-04-03T16:16:42.3583067Z 1f393e236d25: Pulling fs layer 2024-04-03T16:16:42.3583537Z 60489ca1c1f7: Pulling fs layer 2024-04-03T16:16:42.3583965Z 7715b3f284b5: Waiting 2024-04-03T16:16:42.3584353Z fba378c7a893: Waiting 2024-04-03T16:16:42.3584738Z 1f393e236d25: Waiting 2024-04-03T16:16:42.3585100Z 60489ca1c1f7: Waiting 2024-04-03T16:16:42.5070123Z 224fe954d725: Verifying Checksum 2024-04-03T16:16:42.5070866Z 224fe954d725: Download complete 2024-04-03T16:16:42.6313962Z d527cbbb87e3: Verifying Checksum 2024-04-03T16:16:42.6314575Z d527cbbb87e3: Download complete 2024-04-03T16:16:42.7126779Z 7a2c55901189: Verifying Checksum 2024-04-03T16:16:42.7127357Z 7a2c55901189: Download complete 2024-04-03T16:16:42.7270385Z b57676e46aee: Download complete 2024-04-03T16:16:42.8158428Z a41a8d1c11c8: Verifying Checksum 2024-04-03T16:16:42.8986545Z 0c1227890755: Download complete 2024-04-03T16:16:42.9677657Z 75722010b82e: Verifying Checksum 2024-04-03T16:16:42.9678686Z 75722010b82e: Download complete 2024-04-03T16:16:42.9803891Z d8d1234baab3: Verifying Checksum 2024-04-03T16:16:42.9805310Z d8d1234baab3: Download complete 2024-04-03T16:16:43.0623616Z ec1e7978c1fe: Verifying Checksum 2024-04-03T16:16:43.0624220Z ec1e7978c1fe: Download complete 2024-04-03T16:16:43.1515484Z 3704f7764080: Verifying Checksum 2024-04-03T16:16:43.1516254Z 3704f7764080: Download complete 2024-04-03T16:16:43.5232873Z 7a2c55901189: Pull complete 2024-04-03T16:16:43.8649483Z 224fe954d725: Pull complete 2024-04-03T16:16:45.0057131Z 75722010b82e: Pull complete 2024-04-03T16:16:45.1200434Z d527cbbb87e3: Pull complete 2024-04-03T16:16:45.2239844Z b57676e46aee: Pull complete 2024-04-03T16:16:45.6806164Z a24efd49f0c8: Verifying Checksum 2024-04-03T16:16:45.6806654Z a24efd49f0c8: Download complete 2024-04-03T16:16:45.7669352Z 7d4d2c342f82: Verifying Checksum 2024-04-03T16:16:45.7669784Z 7d4d2c342f82: Download complete 2024-04-03T16:16:45.8566922Z 6a9e70a88261: Verifying Checksum 2024-04-03T16:16:45.8568106Z 6a9e70a88261: Download complete 2024-04-03T16:16:45.9477019Z 1173f47dcf05: Verifying Checksum 2024-04-03T16:16:45.9478149Z 1173f47dcf05: Download complete 2024-04-03T16:16:46.9307444Z 325d4583fff5: Verifying Checksum 2024-04-03T16:16:46.9308550Z 325d4583fff5: Download complete 2024-04-03T16:16:47.0213805Z c0d6db5efa7d: Verifying Checksum 2024-04-03T16:16:47.0214720Z c0d6db5efa7d: Download complete 2024-04-03T16:16:47.1066281Z 75830f3a12e2: Verifying Checksum 2024-04-03T16:16:47.1067305Z 75830f3a12e2: Download complete 2024-04-03T16:16:47.1807874Z e857029fb401: Download complete 2024-04-03T16:16:56.0682086Z a8c1e85b5e14: Verifying Checksum 2024-04-03T16:16:56.0682706Z a8c1e85b5e14: Download complete 2024-04-03T16:16:56.1462666Z 3bcad833b4fc: Verifying Checksum 2024-04-03T16:16:56.1463833Z 3bcad833b4fc: Download complete 2024-04-03T16:16:56.2247800Z 11bbf370b108: Download complete 2024-04-03T16:16:56.3083258Z e7825e6a30f3: Download complete 2024-04-03T16:16:56.3861937Z ea7ce2e8ea6d: Verifying Checksum 2024-04-03T16:16:56.3862605Z ea7ce2e8ea6d: Download complete 2024-04-03T16:16:56.4913969Z 73368bd9e761: Verifying Checksum 2024-04-03T16:16:56.4915111Z 73368bd9e761: Download complete 2024-04-03T16:16:57.7735606Z c7972c3f0de1: Verifying Checksum 2024-04-03T16:16:57.7736731Z c7972c3f0de1: Download complete 2024-04-03T16:16:57.8564816Z 3261cbfdbc87: Verifying Checksum 2024-04-03T16:16:57.8565800Z 3261cbfdbc87: Download complete 2024-04-03T16:16:57.9574829Z bbb4242b1c5a: Verifying Checksum 2024-04-03T16:16:57.9576088Z bbb4242b1c5a: Download complete 2024-04-03T16:16:58.0716709Z 6e5a2ed18955: Download complete 2024-04-03T16:16:58.1502871Z c40fcbdc7122: Verifying Checksum 2024-04-03T16:16:58.1503533Z c40fcbdc7122: Download complete 2024-04-03T16:16:58.2422428Z d955fef15356: Verifying Checksum 2024-04-03T16:16:58.2423313Z d955fef15356: Download complete 2024-04-03T16:17:02.7286466Z 2de9ee0e31aa: Verifying Checksum 2024-04-03T16:17:02.7287396Z 2de9ee0e31aa: Download complete 2024-04-03T16:17:02.8717164Z 91caa7e72a27: Verifying Checksum 2024-04-03T16:17:02.8718191Z 91caa7e72a27: Download complete 2024-04-03T16:17:02.9612232Z 0d897f9ab511: Verifying Checksum 2024-04-03T16:17:02.9612932Z 0d897f9ab511: Download complete 2024-04-03T16:17:03.3573243Z 76a6353e12ba: Verifying Checksum 2024-04-03T16:17:03.3573891Z 76a6353e12ba: Download complete 2024-04-03T16:17:03.4359901Z d27855b01494: Verifying Checksum 2024-04-03T16:17:03.4360967Z d27855b01494: Download complete 2024-04-03T16:17:03.5368769Z 04a1e7a30940: Verifying Checksum 2024-04-03T16:17:03.5369621Z 04a1e7a30940: Download complete 2024-04-03T16:17:03.8160510Z 2d6278964762: Verifying Checksum 2024-04-03T16:17:03.8161572Z 2d6278964762: Download complete 2024-04-03T16:17:03.9138312Z 87347beefd8e: Verifying Checksum 2024-04-03T16:17:03.9138890Z 87347beefd8e: Download complete 2024-04-03T16:17:03.9913407Z 4fadf75753a0: Verifying Checksum 2024-04-03T16:17:03.9914521Z 4fadf75753a0: Download complete 2024-04-03T16:17:04.1143540Z 695c95e1ff7f: Download complete 2024-04-03T16:17:04.1242000Z 4f4fb700ef54: Verifying Checksum 2024-04-03T16:17:04.1242716Z 4f4fb700ef54: Download complete 2024-04-03T16:17:04.2073395Z 0c21214a7be6: Verifying Checksum 2024-04-03T16:17:04.2073972Z 0c21214a7be6: Download complete 2024-04-03T16:17:04.2790350Z bde926021ef3: Verifying Checksum 2024-04-03T16:17:04.2791421Z bde926021ef3: Download complete 2024-04-03T16:17:04.8065989Z fa513a4972bd: Verifying Checksum 2024-04-03T16:17:04.8066598Z fa513a4972bd: Download complete 2024-04-03T16:17:04.8963408Z fbb14e703cd0: Verifying Checksum 2024-04-03T16:17:04.8964030Z fbb14e703cd0: Download complete 2024-04-03T16:17:04.9721093Z 2f633f5b71fa: Verifying Checksum 2024-04-03T16:17:04.9721780Z 2f633f5b71fa: Download complete 2024-04-03T16:17:05.0677261Z 939366b420fc: Download complete 2024-04-03T16:17:05.3638593Z aac618ecc588: Download complete 2024-04-03T16:17:08.3311636Z 7ed32bc8e469: Verifying Checksum 2024-04-03T16:17:08.3313189Z 7ed32bc8e469: Download complete 2024-04-03T16:17:08.4130435Z 97cd19635539: Verifying Checksum 2024-04-03T16:17:08.4131636Z 97cd19635539: Download complete 2024-04-03T16:17:08.5195242Z 189619a5dde8: Verifying Checksum 2024-04-03T16:17:08.6772952Z 1ffff58a595c: Download complete 2024-04-03T16:17:08.7596030Z bf176a055b2c: Verifying Checksum 2024-04-03T16:17:08.8492500Z 4fe2333aa4a1: Verifying Checksum 2024-04-03T16:17:08.8493586Z 4fe2333aa4a1: Download complete 2024-04-03T16:17:08.9202806Z d40af861237a: Download complete 2024-04-03T16:17:09.0023995Z a8168f648511: Download complete 2024-04-03T16:17:09.1530502Z 79c82d9080d8: Verifying Checksum 2024-04-03T16:17:09.1531522Z 79c82d9080d8: Download complete 2024-04-03T16:17:09.2440246Z 48bd50555255: Verifying Checksum 2024-04-03T16:17:09.2441327Z 48bd50555255: Download complete 2024-04-03T16:17:09.8532733Z 69bf265ec18d: Verifying Checksum 2024-04-03T16:17:09.8533448Z 69bf265ec18d: Download complete 2024-04-03T16:17:09.9347909Z 076ea06db0f9: Verifying Checksum 2024-04-03T16:17:09.9348539Z 076ea06db0f9: Download complete 2024-04-03T16:17:11.9360769Z a8c1e85b5e14: Pull complete 2024-04-03T16:17:12.0313369Z a41a8d1c11c8: Pull complete 2024-04-03T16:17:12.1515672Z 0c1227890755: Pull complete 2024-04-03T16:17:12.2720699Z d8d1234baab3: Pull complete 2024-04-03T16:17:15.0003888Z 9fc6458e8263: Verifying Checksum 2024-04-03T16:17:15.0004439Z 9fc6458e8263: Download complete 2024-04-03T16:17:15.0914776Z fba378c7a893: Verifying Checksum 2024-04-03T16:17:15.0915411Z fba378c7a893: Download complete 2024-04-03T16:17:15.1760499Z 7715b3f284b5: Verifying Checksum 2024-04-03T16:17:15.1761639Z 7715b3f284b5: Download complete 2024-04-03T16:17:15.7084349Z 1f393e236d25: Verifying Checksum 2024-04-03T16:17:15.7084982Z 1f393e236d25: Download complete 2024-04-03T16:17:15.7948316Z 60489ca1c1f7: Download complete 2024-04-03T16:17:22.4771642Z f884b3670575: Verifying Checksum 2024-04-03T16:17:22.4772341Z f884b3670575: Download complete 2024-04-03T16:17:24.9608363Z acc43aa33d6b: Verifying Checksum 2024-04-03T16:17:24.9608984Z acc43aa33d6b: Download complete 2024-04-03T16:17:48.0854923Z 7ed32bc8e469: Pull complete 2024-04-03T16:17:49.9284627Z ec1e7978c1fe: Pull complete 2024-04-03T16:17:51.6125128Z 3704f7764080: Pull complete 2024-04-03T16:18:00.7475211Z a24efd49f0c8: Pull complete 2024-04-03T16:18:02.8492865Z 7d4d2c342f82: Pull complete 2024-04-03T16:18:04.8183987Z 6a9e70a88261: Pull complete 2024-04-03T16:18:06.6961040Z 1173f47dcf05: Pull complete 2024-04-03T16:18:11.1620103Z 325d4583fff5: Pull complete 2024-04-03T16:18:13.3979703Z c0d6db5efa7d: Pull complete 2024-04-03T16:18:15.2456560Z 75830f3a12e2: Pull complete 2024-04-03T16:18:17.0932599Z e857029fb401: Pull complete 2024-04-03T16:19:07.1782288Z 9fc6458e8263: Pull complete 2024-04-03T16:19:09.0505562Z 3bcad833b4fc: Pull complete 2024-04-03T16:19:11.1143309Z 11bbf370b108: Pull complete 2024-04-03T16:19:12.9590624Z e7825e6a30f3: Pull complete 2024-04-03T16:19:14.8037844Z ea7ce2e8ea6d: Pull complete 2024-04-03T16:19:16.6745867Z 73368bd9e761: Pull complete 2024-04-03T16:19:22.4969296Z c7972c3f0de1: Pull complete 2024-04-03T16:19:24.0694770Z 3261cbfdbc87: Pull complete 2024-04-03T16:19:27.0917173Z bbb4242b1c5a: Pull complete 2024-04-03T16:19:30.2782631Z 6e5a2ed18955: Pull complete 2024-04-03T16:19:32.5980895Z c40fcbdc7122: Pull complete 2024-04-03T16:19:34.4751814Z d955fef15356: Pull complete 2024-04-03T16:19:44.2926794Z 2de9ee0e31aa: Pull complete 2024-04-03T16:19:46.1707407Z 91caa7e72a27: Pull complete 2024-04-03T16:19:48.3075674Z 0d897f9ab511: Pull complete 2024-04-03T16:19:51.2477073Z 76a6353e12ba: Pull complete 2024-04-03T16:19:53.0946127Z d27855b01494: Pull complete 2024-04-03T16:19:54.5467209Z 04a1e7a30940: Pull complete 2024-04-03T16:19:56.8373792Z 2d6278964762: Pull complete 2024-04-03T16:19:58.7113382Z 87347beefd8e: Pull complete 2024-04-03T16:19:59.9716605Z 4fadf75753a0: Pull complete 2024-04-03T16:20:00.0557050Z 695c95e1ff7f: Pull complete 2024-04-03T16:20:00.1686772Z 4f4fb700ef54: Pull complete 2024-04-03T16:20:00.2674158Z 0c21214a7be6: Pull complete 2024-04-03T16:20:00.3553948Z bde926021ef3: Pull complete 2024-04-03T16:20:01.7650335Z fa513a4972bd: Pull complete 2024-04-03T16:20:01.8721963Z fbb14e703cd0: Pull complete 2024-04-03T16:20:01.9640990Z 2f633f5b71fa: Pull complete 2024-04-03T16:20:02.1560806Z 939366b420fc: Pull complete 2024-04-03T16:20:02.2564475Z aac618ecc588: Pull complete 2024-04-03T16:20:37.0031393Z f884b3670575: Pull complete 2024-04-03T16:20:39.3511597Z 97cd19635539: Pull complete 2024-04-03T16:20:41.2898810Z 189619a5dde8: Pull complete 2024-04-03T16:20:43.3567954Z 1ffff58a595c: Pull complete 2024-04-03T16:20:45.2343913Z bf176a055b2c: Pull complete 2024-04-03T16:20:47.1054215Z 4fe2333aa4a1: Pull complete 2024-04-03T16:20:48.9821029Z d40af861237a: Pull complete 2024-04-03T16:20:50.8583530Z a8168f648511: Pull complete 2024-04-03T16:20:53.7145739Z 79c82d9080d8: Pull complete 2024-04-03T16:20:55.4630527Z 48bd50555255: Pull complete 2024-04-03T16:20:59.4371555Z 69bf265ec18d: Pull complete 2024-04-03T16:21:01.3128609Z 076ea06db0f9: Pull complete 2024-04-03T16:21:20.6372071Z acc43aa33d6b: Pull complete 2024-04-03T16:21:20.7327387Z fba378c7a893: Pull complete 2024-04-03T16:21:20.8320439Z 7715b3f284b5: Pull complete 2024-04-03T16:21:21.6882290Z 1f393e236d25: Pull complete 2024-04-03T16:21:21.7930353Z 60489ca1c1f7: Pull complete 2024-04-03T16:21:22.1148893Z Digest: sha256:a560526b69e9ece9c90f2a23812ba2f1a57591b7cd437a945ebbb84c42e75301 2024-04-03T16:21:22.1179405Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:21:22.1205990Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:21:22.1393819Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-04-03T16:21:22.1394545Z with: 2024-04-03T16:21:22.1394897Z driver-version: 535.54.03 2024-04-03T16:21:22.1395322Z env: 2024-04-03T16:21:22.1395639Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:21:22.1396073Z ##[endgroup] 2024-04-03T16:21:22.1451506Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-04-03T16:21:22.1452210Z with: 2024-04-03T16:21:22.1452553Z timeout_minutes: 10 2024-04-03T16:21:22.1452961Z max_attempts: 3 2024-04-03T16:21:22.1493251Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2024-04-03T16:21:22.1534031Z retry_wait_seconds: 10 2024-04-03T16:21:22.1534474Z polling_interval_seconds: 1 2024-04-03T16:21:22.1534941Z warning_on_retry: true 2024-04-03T16:21:22.1535378Z continue_on_error: false 2024-04-03T16:21:22.1535774Z env: 2024-04-03T16:21:22.1536109Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:21:22.1536553Z DRIVER_VERSION: 535.54.03 2024-04-03T16:21:22.1536963Z ##[endgroup] 2024-04-03T16:21:22.2302977Z == Installing nvidia driver NVIDIA-Linux-x86_64-535.54.03.run == 2024-04-03T16:21:22.2304629Z + pre_install_nvidia_driver_amzn2 2024-04-03T16:21:22.2305323Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-04-03T16:21:22.7486455Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-04-03T16:21:22.8085272Z No Match for argument: nvidia-driver-latest-dkms 2024-04-03T16:21:22.8464246Z No Packages marked for removal 2024-04-03T16:21:22.8622955Z + install_nvidia_driver_common 2024-04-03T16:21:22.8627016Z + echo 'Before installing NVIDIA driver' 2024-04-03T16:21:22.8627626Z + lspci 2024-04-03T16:21:22.8629327Z Before installing NVIDIA driver 2024-04-03T16:21:22.8803753Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-04-03T16:21:22.8804775Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-04-03T16:21:22.8805790Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-04-03T16:21:22.8806747Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-04-03T16:21:22.8807574Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-04-03T16:21:22.8808452Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-04-03T16:21:22.8809500Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-04-03T16:21:22.8810537Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-04-03T16:21:22.8811286Z + lsmod 2024-04-03T16:21:22.8827457Z Module Size Used by 2024-04-03T16:21:22.8827985Z nvidia_modeset 1306624 0 2024-04-03T16:21:22.8828449Z veth 16384 0 2024-04-03T16:21:22.8828908Z nvidia_uvm 1466368 0 2024-04-03T16:21:22.8829471Z nvidia 56422400 2 nvidia_uvm,nvidia_modeset 2024-04-03T16:21:22.8830068Z drm 421888 1 nvidia 2024-04-03T16:21:22.8830595Z i2c_core 77824 2 nvidia,drm 2024-04-03T16:21:22.8831273Z backlight 16384 1 nvidia_modeset 2024-04-03T16:21:22.8832311Z xt_conntrack 16384 1 2024-04-03T16:21:22.8833731Z ipt_MASQUERADE 16384 1 2024-04-03T16:21:22.8834444Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-04-03T16:21:22.8835017Z nf_conntrack_netlink 49152 0 2024-04-03T16:21:22.8835599Z nfnetlink 16384 2 nf_conntrack_netlink 2024-04-03T16:21:22.8836189Z xfrm_user 45056 1 2024-04-03T16:21:22.8836658Z xfrm_algo 16384 1 xfrm_user 2024-04-03T16:21:22.8837178Z iptable_nat 16384 1 2024-04-03T16:21:22.8837673Z nf_conntrack_ipv4 16384 3 2024-04-03T16:21:22.8838195Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-04-03T16:21:22.8838798Z nf_nat_ipv4 16384 1 iptable_nat 2024-04-03T16:21:22.8840079Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-04-03T16:21:22.8842591Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-04-03T16:21:22.8844601Z xt_addrtype 16384 2 2024-04-03T16:21:22.8845251Z iptable_filter 16384 1 2024-04-03T16:21:22.8846331Z br_netfilter 24576 0 2024-04-03T16:21:22.8846901Z bridge 172032 1 br_netfilter 2024-04-03T16:21:22.8847576Z stp 16384 1 bridge 2024-04-03T16:21:22.8848720Z llc 16384 2 bridge,stp 2024-04-03T16:21:22.8849641Z overlay 86016 0 2024-04-03T16:21:22.8850111Z sunrpc 393216 1 2024-04-03T16:21:22.8850807Z dm_mirror 28672 0 2024-04-03T16:21:22.8851283Z dm_region_hash 20480 1 dm_mirror 2024-04-03T16:21:22.8851896Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-04-03T16:21:22.8852630Z dm_mod 143360 2 dm_log,dm_mirror 2024-04-03T16:21:22.8853845Z dax 69632 1 dm_mod 2024-04-03T16:21:22.8854389Z sb_edac 24576 0 2024-04-03T16:21:22.8854854Z crc32_pclmul 16384 0 2024-04-03T16:21:22.8855327Z ghash_clmulni_intel 16384 0 2024-04-03T16:21:22.8855791Z pcbc 16384 0 2024-04-03T16:21:22.8856261Z aesni_intel 188416 0 2024-04-03T16:21:22.8856722Z mousedev 24576 0 2024-04-03T16:21:22.8857195Z aes_x86_64 20480 1 aesni_intel 2024-04-03T16:21:22.8857757Z crypto_simd 16384 1 aesni_intel 2024-04-03T16:21:22.8858321Z glue_helper 16384 1 aesni_intel 2024-04-03T16:21:22.8859005Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-04-03T16:21:22.8859698Z psmouse 32768 0 2024-04-03T16:21:22.8860154Z evdev 20480 3 2024-04-03T16:21:22.8860597Z pcc_cpufreq 16384 0 2024-04-03T16:21:22.8861058Z button 16384 0 2024-04-03T16:21:22.8861511Z ena 139264 0 2024-04-03T16:21:22.8861950Z ptp 20480 1 ena 2024-04-03T16:21:22.8862593Z pps_core 20480 1 ptp 2024-04-03T16:21:22.8863083Z ata_piix 36864 0 2024-04-03T16:21:22.8863672Z libata 266240 1 ata_piix 2024-04-03T16:21:22.8864191Z xen_blkfront 49152 2 2024-04-03T16:21:22.8864656Z crc32c_intel 24576 0 2024-04-03T16:21:22.8865113Z scsi_mod 245760 1 libata 2024-04-03T16:21:22.8865609Z autofs4 49152 2 2024-04-03T16:21:22.8866057Z + modinfo nvidia 2024-04-03T16:21:22.8866876Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-04-03T16:21:22.8867758Z firmware: nvidia/535.54.03/gsp_tu10x.bin 2024-04-03T16:21:22.8868372Z firmware: nvidia/535.54.03/gsp_ga10x.bin 2024-04-03T16:21:22.8868976Z alias: char-major-195-* 2024-04-03T16:21:22.8869455Z version: 535.54.03 2024-04-03T16:21:22.8869883Z supported: external 2024-04-03T16:21:22.8870292Z license: NVIDIA 2024-04-03T16:21:22.8870730Z srcversion: EA9C7EF32617E104C8240C4 2024-04-03T16:21:22.8871319Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-04-03T16:21:22.8872091Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-04-03T16:21:22.8872739Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-04-03T16:21:22.8873358Z depends: i2c-core,drm 2024-04-03T16:21:22.8873785Z retpoline: Y 2024-04-03T16:21:22.8874164Z name: nvidia 2024-04-03T16:21:22.8874868Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-04-03T16:21:22.8875679Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-04-03T16:21:22.8876499Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-04-03T16:21:22.8877266Z parm: NVreg_ResmanDebugLevel:int 2024-04-03T16:21:22.8877805Z parm: NVreg_RmLogonRC:int 2024-04-03T16:21:22.8878342Z parm: NVreg_ModifyDeviceFiles:int 2024-04-03T16:21:22.8878914Z parm: NVreg_DeviceFileUID:int 2024-04-03T16:21:22.8880258Z parm: NVreg_DeviceFileGID:int 2024-04-03T16:21:22.8880819Z parm: NVreg_DeviceFileMode:int 2024-04-03T16:21:22.8881492Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-04-03T16:21:22.8882190Z parm: NVreg_UsePageAttributeTable:int 2024-04-03T16:21:22.8882794Z parm: NVreg_EnablePCIeGen3:int 2024-04-03T16:21:22.8883336Z parm: NVreg_EnableMSI:int 2024-04-03T16:21:22.8883835Z parm: NVreg_TCEBypassMode:int 2024-04-03T16:21:22.8884408Z parm: NVreg_EnableStreamMemOPs:int 2024-04-03T16:21:22.8885076Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-04-03T16:21:22.8885974Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-04-03T16:21:22.8886684Z parm: NVreg_EnableS0ixPowerManagement:int 2024-04-03T16:21:22.8887458Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-04-03T16:21:22.8888198Z parm: NVreg_DynamicPowerManagement:int 2024-04-03T16:21:22.8889023Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-04-03T16:21:22.8889778Z parm: NVreg_EnableGpuFirmware:int 2024-04-03T16:21:22.8890404Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-04-03T16:21:22.8891056Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-04-03T16:21:22.8891742Z parm: NVreg_EnableUserNUMAManagement:int 2024-04-03T16:21:22.8892369Z parm: NVreg_MemoryPoolSize:int 2024-04-03T16:21:22.8892934Z parm: NVreg_KMallocHeapMaxSize:int 2024-04-03T16:21:22.8893538Z parm: NVreg_VMallocHeapMaxSize:int 2024-04-03T16:21:22.8894129Z parm: NVreg_IgnoreMMIOCheck:int 2024-04-03T16:21:22.8894681Z parm: NVreg_NvLinkDisable:int 2024-04-03T16:21:22.8895313Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-04-03T16:21:22.8895977Z parm: NVreg_RegisterPCIDriver:int 2024-04-03T16:21:22.8896564Z parm: NVreg_EnableResizableBar:int 2024-04-03T16:21:22.8897197Z parm: NVreg_EnableDbgBreakpoint:int 2024-04-03T16:21:22.8897802Z parm: NVreg_RegistryDwords:charp 2024-04-03T16:21:22.8898406Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-04-03T16:21:22.8899013Z parm: NVreg_RmMsg:charp 2024-04-03T16:21:22.8899532Z parm: NVreg_GpuBlacklist:charp 2024-04-03T16:21:22.8900101Z parm: NVreg_TemporaryFilePath:charp 2024-04-03T16:21:22.8900690Z parm: NVreg_ExcludedGpus:charp 2024-04-03T16:21:22.8901257Z parm: NVreg_DmaRemapPeerMmio:int 2024-04-03T16:21:22.8901839Z parm: NVreg_RmNvlinkBandwidth:charp 2024-04-03T16:21:22.8902426Z parm: rm_firmware_active:charp 2024-04-03T16:21:22.8902953Z + HAS_NVIDIA_DRIVER=0 2024-04-03T16:21:22.8903418Z ++ command -v nvidia-smi 2024-04-03T16:21:22.8903917Z + '[' -x /usr/bin/nvidia-smi ']' 2024-04-03T16:21:22.8904372Z + set +e 2024-04-03T16:21:22.8904984Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-04-03T16:21:23.7212988Z + INSTALLED_DRIVER_VERSION=535.54.03 2024-04-03T16:21:23.7214190Z + NVIDIA_SMI_STATUS=0 2024-04-03T16:21:23.7215289Z + '[' 0 -ne 0 ']' 2024-04-03T16:21:23.7216046Z + '[' 535.54.03 '!=' 535.54.03 ']' 2024-04-03T16:21:23.7216864Z + HAS_NVIDIA_DRIVER=1 2024-04-03T16:21:23.7217918Z + echo 'NVIDIA driver (535.54.03) has already been installed. Skipping NVIDIA driver installation' 2024-04-03T16:21:23.7219652Z + set -e 2024-04-03T16:21:23.7220026Z + '[' 1 -eq 0 ']' 2024-04-03T16:21:23.7220750Z NVIDIA driver (535.54.03) has already been installed. Skipping NVIDIA driver installation 2024-04-03T16:21:23.7221608Z + post_install_nvidia_driver_common 2024-04-03T16:21:23.7222110Z + sudo modprobe nvidia 2024-04-03T16:21:23.7373237Z + echo 'After installing NVIDIA driver' 2024-04-03T16:21:23.7374534Z + lspci 2024-04-03T16:21:23.7375275Z After installing NVIDIA driver 2024-04-03T16:21:23.7549665Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-04-03T16:21:23.7551355Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-04-03T16:21:23.7552340Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-04-03T16:21:23.7553303Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-04-03T16:21:23.7554126Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-04-03T16:21:23.7554978Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-04-03T16:21:23.7556025Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-04-03T16:21:23.7557076Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-04-03T16:21:23.7558074Z + lsmod 2024-04-03T16:21:23.7572951Z Module Size Used by 2024-04-03T16:21:23.7573986Z nvidia_modeset 1306624 0 2024-04-03T16:21:23.7574958Z veth 16384 0 2024-04-03T16:21:23.7575772Z nvidia_uvm 1466368 0 2024-04-03T16:21:23.7576334Z nvidia 56422400 2 nvidia_uvm,nvidia_modeset 2024-04-03T16:21:23.7577130Z drm 421888 1 nvidia 2024-04-03T16:21:23.7578194Z i2c_core 77824 2 nvidia,drm 2024-04-03T16:21:23.7579339Z backlight 16384 1 nvidia_modeset 2024-04-03T16:21:23.7580163Z xt_conntrack 16384 1 2024-04-03T16:21:23.7580629Z ipt_MASQUERADE 16384 1 2024-04-03T16:21:23.7581175Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-04-03T16:21:23.7581759Z nf_conntrack_netlink 49152 0 2024-04-03T16:21:23.7582292Z nfnetlink 16384 2 nf_conntrack_netlink 2024-04-03T16:21:23.7582899Z xfrm_user 45056 1 2024-04-03T16:21:23.7583382Z xfrm_algo 16384 1 xfrm_user 2024-04-03T16:21:23.7583911Z iptable_nat 16384 1 2024-04-03T16:21:23.7584363Z nf_conntrack_ipv4 16384 3 2024-04-03T16:21:23.7584896Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-04-03T16:21:23.7585494Z nf_nat_ipv4 16384 1 iptable_nat 2024-04-03T16:21:23.7586127Z nf_nat 32768 2 nf_nat_masquerade_ipv4,nf_nat_ipv4 2024-04-03T16:21:23.7588301Z nf_conntrack 155648 7 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_netlink 2024-04-03T16:21:23.7590802Z xt_addrtype 16384 2 2024-04-03T16:21:23.7591725Z iptable_filter 16384 1 2024-04-03T16:21:23.7592206Z br_netfilter 24576 0 2024-04-03T16:21:23.7592710Z bridge 172032 1 br_netfilter 2024-04-03T16:21:23.7593237Z stp 16384 1 bridge 2024-04-03T16:21:23.7593757Z llc 16384 2 bridge,stp 2024-04-03T16:21:23.7594274Z overlay 86016 0 2024-04-03T16:21:23.7594725Z sunrpc 393216 1 2024-04-03T16:21:23.7595184Z dm_mirror 28672 0 2024-04-03T16:21:23.7595669Z dm_region_hash 20480 1 dm_mirror 2024-04-03T16:21:23.7596253Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-04-03T16:21:23.7596882Z dm_mod 143360 2 dm_log,dm_mirror 2024-04-03T16:21:23.7597435Z dax 69632 1 dm_mod 2024-04-03T16:21:23.7597903Z sb_edac 24576 0 2024-04-03T16:21:23.7598358Z crc32_pclmul 16384 0 2024-04-03T16:21:23.7599007Z ghash_clmulni_intel 16384 0 2024-04-03T16:21:23.7599695Z pcbc 16384 0 2024-04-03T16:21:23.7600158Z aesni_intel 188416 0 2024-04-03T16:21:23.7600615Z mousedev 24576 0 2024-04-03T16:21:23.7601177Z aes_x86_64 20480 1 aesni_intel 2024-04-03T16:21:23.7601888Z crypto_simd 16384 1 aesni_intel 2024-04-03T16:21:23.7602457Z glue_helper 16384 1 aesni_intel 2024-04-03T16:21:23.7603144Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-04-03T16:21:23.7603819Z psmouse 32768 0 2024-04-03T16:21:23.7604271Z evdev 20480 3 2024-04-03T16:21:23.7604710Z pcc_cpufreq 16384 0 2024-04-03T16:21:23.7605166Z button 16384 0 2024-04-03T16:21:23.7605615Z ena 139264 0 2024-04-03T16:21:23.7606052Z ptp 20480 1 ena 2024-04-03T16:21:23.7606538Z pps_core 20480 1 ptp 2024-04-03T16:21:23.7607022Z ata_piix 36864 0 2024-04-03T16:21:23.7607484Z libata 266240 1 ata_piix 2024-04-03T16:21:23.7607997Z xen_blkfront 49152 2 2024-04-03T16:21:23.7608450Z crc32c_intel 24576 0 2024-04-03T16:21:23.7608901Z scsi_mod 245760 1 libata 2024-04-03T16:21:23.7609391Z autofs4 49152 2 2024-04-03T16:21:23.7609834Z + modinfo nvidia 2024-04-03T16:21:23.7610701Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-04-03T16:21:23.7611732Z firmware: nvidia/535.54.03/gsp_tu10x.bin 2024-04-03T16:21:23.7612337Z firmware: nvidia/535.54.03/gsp_ga10x.bin 2024-04-03T16:21:23.7612965Z alias: char-major-195-* 2024-04-03T16:21:23.7613421Z version: 535.54.03 2024-04-03T16:21:23.7613846Z supported: external 2024-04-03T16:21:23.7614265Z license: NVIDIA 2024-04-03T16:21:23.7614685Z srcversion: EA9C7EF32617E104C8240C4 2024-04-03T16:21:23.7615276Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-04-03T16:21:23.7615906Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-04-03T16:21:23.7616497Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-04-03T16:21:23.7617109Z depends: i2c-core,drm 2024-04-03T16:21:23.7617549Z retpoline: Y 2024-04-03T16:21:23.7617906Z name: nvidia 2024-04-03T16:21:23.7618598Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-04-03T16:21:23.7619425Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-04-03T16:21:23.7620230Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-04-03T16:21:23.7620978Z parm: NVreg_ResmanDebugLevel:int 2024-04-03T16:21:23.7621530Z parm: NVreg_RmLogonRC:int 2024-04-03T16:21:23.7622050Z parm: NVreg_ModifyDeviceFiles:int 2024-04-03T16:21:23.7622617Z parm: NVreg_DeviceFileUID:int 2024-04-03T16:21:23.7623155Z parm: NVreg_DeviceFileGID:int 2024-04-03T16:21:23.7623685Z parm: NVreg_DeviceFileMode:int 2024-04-03T16:21:23.7624342Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-04-03T16:21:23.7625047Z parm: NVreg_UsePageAttributeTable:int 2024-04-03T16:21:23.7625631Z parm: NVreg_EnablePCIeGen3:int 2024-04-03T16:21:23.7626161Z parm: NVreg_EnableMSI:int 2024-04-03T16:21:23.7626675Z parm: NVreg_TCEBypassMode:int 2024-04-03T16:21:23.7627228Z parm: NVreg_EnableStreamMemOPs:int 2024-04-03T16:21:23.7627899Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-04-03T16:21:23.7628631Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-04-03T16:21:23.7629316Z parm: NVreg_EnableS0ixPowerManagement:int 2024-04-03T16:21:23.7630082Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-04-03T16:21:23.7630834Z parm: NVreg_DynamicPowerManagement:int 2024-04-03T16:21:23.7631602Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-04-03T16:21:23.7632465Z parm: NVreg_EnableGpuFirmware:int 2024-04-03T16:21:23.7633105Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-04-03T16:21:23.7633770Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-04-03T16:21:23.7634435Z parm: NVreg_EnableUserNUMAManagement:int 2024-04-03T16:21:23.7635053Z parm: NVreg_MemoryPoolSize:int 2024-04-03T16:21:23.7635629Z parm: NVreg_KMallocHeapMaxSize:int 2024-04-03T16:21:23.7636211Z parm: NVreg_VMallocHeapMaxSize:int 2024-04-03T16:21:23.7636801Z parm: NVreg_IgnoreMMIOCheck:int 2024-04-03T16:21:23.7637355Z parm: NVreg_NvLinkDisable:int 2024-04-03T16:21:23.7637965Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-04-03T16:21:23.7638620Z parm: NVreg_RegisterPCIDriver:int 2024-04-03T16:21:23.7639463Z parm: NVreg_EnableResizableBar:int 2024-04-03T16:21:23.7640067Z parm: NVreg_EnableDbgBreakpoint:int 2024-04-03T16:21:23.7640664Z parm: NVreg_RegistryDwords:charp 2024-04-03T16:21:23.7641296Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-04-03T16:21:23.7641882Z parm: NVreg_RmMsg:charp 2024-04-03T16:21:23.7642386Z parm: NVreg_GpuBlacklist:charp 2024-04-03T16:21:23.7642964Z parm: NVreg_TemporaryFilePath:charp 2024-04-03T16:21:23.7643531Z parm: NVreg_ExcludedGpus:charp 2024-04-03T16:21:23.7644097Z parm: NVreg_DmaRemapPeerMmio:int 2024-04-03T16:21:23.7644694Z parm: NVreg_RmNvlinkBandwidth:charp 2024-04-03T16:21:23.7645411Z parm: rm_firmware_active:charp 2024-04-03T16:21:23.7645911Z + set +e 2024-04-03T16:21:23.7646287Z + nvidia-smi 2024-04-03T16:21:24.3231915Z Wed Apr 3 16:21:24 2024 2024-04-03T16:21:24.3233543Z +---------------------------------------------------------------------------------------+ 2024-04-03T16:21:24.3234615Z | NVIDIA-SMI 535.54.03 Driver Version: 535.54.03 CUDA Version: 12.2 | 2024-04-03T16:21:24.3235622Z |-----------------------------------------+----------------------+----------------------+ 2024-04-03T16:21:24.3236623Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-04-03T16:21:24.3237740Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-04-03T16:21:24.3238584Z | | | MIG M. | 2024-04-03T16:21:24.3239530Z |=========================================+======================+======================| 2024-04-03T16:21:24.3278197Z | 0 Tesla M60 Off | 00000000:00:1E.0 Off | 0 | 2024-04-03T16:21:24.3280250Z | N/A 27C P0 38W / 150W | 0MiB / 7680MiB | 1% Default | 2024-04-03T16:21:24.3281538Z | | | N/A | 2024-04-03T16:21:24.3282394Z +-----------------------------------------+----------------------+----------------------+ 2024-04-03T16:21:24.3283142Z 2024-04-03T16:21:24.3283958Z +---------------------------------------------------------------------------------------+ 2024-04-03T16:21:24.3284735Z | Processes: | 2024-04-03T16:21:24.3285626Z | GPU GI CI PID Type Process name GPU Memory | 2024-04-03T16:21:24.3286509Z | ID ID Usage | 2024-04-03T16:21:24.3287195Z |=======================================================================================| 2024-04-03T16:21:24.3288021Z | No running processes found | 2024-04-03T16:21:24.3289022Z +---------------------------------------------------------------------------------------+ 2024-04-03T16:21:25.8677682Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-04-03T16:21:26.4365210Z Tesla M60 2024-04-03T16:21:26.6855328Z + NVIDIA_SMI_STATUS=0 2024-04-03T16:21:26.6855945Z + '[' 0 -eq 0 ']' 2024-04-03T16:21:26.6856421Z + echo 'INFO: Ignoring allowed status 0' 2024-04-03T16:21:26.6856962Z + set -e 2024-04-03T16:21:26.6857204Z INFO: Ignoring allowed status 0 2024-04-03T16:21:26.6862909Z == Installing nvidia container toolkit for amzn2 == 2024-04-03T16:21:26.6866367Z + sudo yum install -y yum-utils 2024-04-03T16:21:27.1871036Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-04-03T16:21:28.9127768Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-04-03T16:21:28.9128665Z Nothing to do 2024-04-03T16:21:29.0648104Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-04-03T16:21:29.5971328Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-04-03T16:21:29.6280321Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-04-03T16:21:29.6281745Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-04-03T16:21:29.6282980Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-04-03T16:21:29.6428009Z + sudo yum install -y nvidia-docker2 2024-04-03T16:21:30.1830888Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-04-03T16:21:31.9286038Z Package nvidia-docker2-2.13.0-1.noarch already installed and latest version 2024-04-03T16:21:31.9287466Z Nothing to do 2024-04-03T16:21:32.0851542Z + sudo systemctl restart docker 2024-04-03T16:21:52.2571142Z Command completed after 1 attempt(s). 2024-04-03T16:21:52.2648759Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-04-03T16:21:52.2649774Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-04-03T16:21:52.2650644Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-04-03T16:21:52.2651466Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-04-03T16:21:52.2660340Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:21:52.2660965Z env: 2024-04-03T16:21:52.2661298Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:21:52.2661842Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:21:52.2662455Z ##[endgroup] 2024-04-03T16:21:52.5623964Z Defaulting to user installation because normal site-packages is not writeable 2024-04-03T16:21:52.5839409Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-04-03T16:21:52.5964520Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-04-03T16:21:52.7169617Z Prepare all required actions 2024-04-03T16:21:52.7170315Z Getting action download info 2024-04-03T16:21:52.8586727Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-04-03T16:21:53.1468768Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-04-03T16:21:53.2810031Z ##[group]Run ./.github/actions/download-build-artifacts 2024-04-03T16:21:53.2810634Z with: 2024-04-03T16:21:53.2811000Z name: linux-focal-cuda12.1-py3.10-gcc9 2024-04-03T16:21:53.2811540Z s3-bucket: gha-artifacts 2024-04-03T16:21:53.2811953Z env: 2024-04-03T16:21:53.2812267Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:21:53.2812828Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:21:53.2813444Z ##[endgroup] 2024-04-03T16:21:53.2857259Z ##[group]Run seemethere/download-artifact-s3@v4 2024-04-03T16:21:53.2857800Z with: 2024-04-03T16:21:53.2858243Z name: linux-focal-cuda12.1-py3.10-gcc9 2024-04-03T16:21:53.2858769Z s3-bucket: gha-artifacts 2024-04-03T16:21:53.2859201Z region: us-east-1 2024-04-03T16:21:53.2859565Z env: 2024-04-03T16:21:53.2859877Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:21:53.2860434Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:21:53.2861033Z ##[endgroup] 2024-04-03T16:21:53.9135356Z (node:128848) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-04-03T16:21:53.9136204Z 2024-04-03T16:21:53.9136529Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-04-03T16:21:53.9137816Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-04-03T16:21:53.9139468Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-04-03T16:21:54.0138555Z Found 1 objects with prefix pytorch/pytorch/8536950919/linux-focal-cuda12.1-py3.10-gcc9/ 2024-04-03T16:21:54.0141187Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-04-03T16:22:06.7520646Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-04-03T16:22:06.7530752Z Artifact download has finished successfully 2024-04-03T16:22:06.7744771Z ##[group]Run unzip -o artifacts.zip 2024-04-03T16:22:06.7745318Z unzip -o artifacts.zip 2024-04-03T16:22:06.7753906Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:22:06.7754537Z env: 2024-04-03T16:22:06.7754877Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:06.7755421Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:06.7756035Z ##[endgroup] 2024-04-03T16:22:06.7794445Z Archive: artifacts.zip 2024-04-03T16:22:06.7796526Z creating: dist/ 2024-04-03T16:22:09.1447924Z inflating: dist/torch-2.4.0a0+gitd253bd7-cp310-cp310-linux_x86_64.whl 2024-04-03T16:22:09.1448802Z creating: build/custom_test_artifacts/ 2024-04-03T16:22:09.1449538Z creating: build/custom_test_artifacts/custom-op-build/ 2024-04-03T16:22:09.1450441Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-04-03T16:22:09.1451506Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-04-03T16:22:09.1458292Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-04-03T16:22:09.1459512Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-04-03T16:22:09.1460707Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-04-03T16:22:09.1461984Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-04-03T16:22:09.1463240Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-04-03T16:22:09.1464672Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-04-03T16:22:09.1466393Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-04-03T16:22:09.1467740Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-04-03T16:22:09.1469020Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-04-03T16:22:09.1470499Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-04-03T16:22:09.1472002Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-04-03T16:22:09.1473476Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-04-03T16:22:09.1474938Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-04-03T16:22:09.1476437Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-04-03T16:22:09.1477950Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-04-03T16:22:09.1479669Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-04-03T16:22:09.1480996Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-04-03T16:22:09.1527596Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-04-03T16:22:09.1576415Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-04-03T16:22:09.1578300Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-04-03T16:22:09.1633923Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-04-03T16:22:09.1635749Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-04-03T16:22:09.1637610Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-04-03T16:22:09.1639805Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-04-03T16:22:09.1641678Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-04-03T16:22:09.1643483Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-04-03T16:22:09.1645296Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-04-03T16:22:09.1647365Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-04-03T16:22:09.1649132Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-04-03T16:22:09.1650796Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-04-03T16:22:09.1652401Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-04-03T16:22:09.1653980Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-04-03T16:22:09.1655573Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-04-03T16:22:09.1657098Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-04-03T16:22:09.1658679Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-04-03T16:22:09.1723336Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-04-03T16:22:09.1798081Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-04-03T16:22:09.1799918Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-04-03T16:22:09.1801207Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-04-03T16:22:09.1802333Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-04-03T16:22:09.1803471Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-04-03T16:22:09.1804659Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-04-03T16:22:09.1805994Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-04-03T16:22:09.1807481Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-04-03T16:22:09.1808914Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-04-03T16:22:09.1810250Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-04-03T16:22:09.1812107Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-04-03T16:22:09.1813487Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-04-03T16:22:09.1814886Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-04-03T16:22:09.1816273Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-04-03T16:22:09.1817659Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-04-03T16:22:09.1834020Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-04-03T16:22:09.2000889Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-04-03T16:22:09.2002179Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-04-03T16:22:09.2003582Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-04-03T16:22:09.2005132Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-04-03T16:22:09.2006642Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-04-03T16:22:09.2008042Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-04-03T16:22:09.2009706Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-04-03T16:22:09.2011189Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-04-03T16:22:09.2012661Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-04-03T16:22:09.2014125Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-04-03T16:22:09.2015544Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-04-03T16:22:09.2033078Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-04-03T16:22:09.2136427Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-04-03T16:22:09.2137975Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-04-03T16:22:09.2139357Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-04-03T16:22:09.2140772Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-04-03T16:22:09.2141984Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-04-03T16:22:09.2143120Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-04-03T16:22:09.2144277Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-04-03T16:22:09.2145370Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-04-03T16:22:09.2146364Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-04-03T16:22:09.2147381Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-04-03T16:22:09.2285431Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-04-03T16:22:09.2363111Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-04-03T16:22:09.2364054Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-04-03T16:22:09.2364929Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-04-03T16:22:09.2365974Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-04-03T16:22:09.2373750Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-04-03T16:22:09.2374926Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-04-03T16:22:09.2376120Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-04-03T16:22:09.2377370Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-04-03T16:22:09.2378597Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-04-03T16:22:09.2379998Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-04-03T16:22:09.2381407Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-04-03T16:22:09.2382691Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-04-03T16:22:09.2383940Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-04-03T16:22:09.2385407Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-04-03T16:22:09.2386894Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-04-03T16:22:09.2388347Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-04-03T16:22:09.2389782Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-04-03T16:22:09.2391482Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-04-03T16:22:09.2393022Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-04-03T16:22:09.2394340Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-04-03T16:22:09.2395607Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-04-03T16:22:09.2442578Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-04-03T16:22:09.2491403Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-04-03T16:22:09.2493307Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-04-03T16:22:09.2548754Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-04-03T16:22:09.2550594Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-04-03T16:22:09.2552627Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-04-03T16:22:09.2554578Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-04-03T16:22:09.2556394Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-04-03T16:22:09.2558181Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-04-03T16:22:09.2560334Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-04-03T16:22:09.2562143Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-04-03T16:22:09.2563883Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-04-03T16:22:09.2565539Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-04-03T16:22:09.2567130Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-04-03T16:22:09.2568690Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-04-03T16:22:09.2570251Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-04-03T16:22:09.2571787Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-04-03T16:22:09.2573345Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-04-03T16:22:09.2636691Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-04-03T16:22:09.2711771Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-04-03T16:22:09.2713294Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-04-03T16:22:09.2714545Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-04-03T16:22:09.2715627Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-04-03T16:22:09.2716784Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-04-03T16:22:09.2717989Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-04-03T16:22:09.2719651Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-04-03T16:22:09.2721426Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-04-03T16:22:09.2722907Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-04-03T16:22:09.2724281Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-04-03T16:22:09.2725720Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-04-03T16:22:09.2727133Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-04-03T16:22:09.2728565Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-04-03T16:22:09.2729991Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-04-03T16:22:09.2731383Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-04-03T16:22:09.2746688Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-04-03T16:22:09.2826357Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-04-03T16:22:09.2827916Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-04-03T16:22:09.2829270Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-04-03T16:22:09.2830505Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-04-03T16:22:09.2831669Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-04-03T16:22:09.2832810Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-04-03T16:22:09.2833948Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-04-03T16:22:09.2835041Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-04-03T16:22:09.2836019Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-04-03T16:22:09.2837030Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-04-03T16:22:09.2897266Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-04-03T16:22:09.2898231Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-04-03T16:22:09.2899197Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-04-03T16:22:09.2900303Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-04-03T16:22:09.2907195Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-04-03T16:22:09.2908480Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-04-03T16:22:09.2909741Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-04-03T16:22:09.2911114Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-04-03T16:22:09.2912451Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-04-03T16:22:09.2913955Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-04-03T16:22:09.2915448Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-04-03T16:22:09.2916814Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-04-03T16:22:09.2918156Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-04-03T16:22:09.2920023Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-04-03T16:22:09.2921849Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-04-03T16:22:09.2923402Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-04-03T16:22:09.2924957Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-04-03T16:22:09.2926528Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-04-03T16:22:09.2928107Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-04-03T16:22:09.2929521Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-04-03T16:22:09.2930896Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-04-03T16:22:09.2975718Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-04-03T16:22:09.3024915Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-04-03T16:22:09.3027036Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-04-03T16:22:09.3082502Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-04-03T16:22:09.3084408Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-04-03T16:22:09.3086356Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-04-03T16:22:09.3088361Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-04-03T16:22:09.3090301Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-04-03T16:22:09.3092197Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-04-03T16:22:09.3094110Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-04-03T16:22:09.3095986Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-04-03T16:22:09.3097831Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-04-03T16:22:09.3099549Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-04-03T16:22:09.3101232Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-04-03T16:22:09.3102897Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-04-03T16:22:09.3104577Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-04-03T16:22:09.3106177Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-04-03T16:22:09.3107825Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-04-03T16:22:09.3170600Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-04-03T16:22:09.3245706Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-04-03T16:22:09.3247303Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-04-03T16:22:09.3248864Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-04-03T16:22:09.3250051Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-04-03T16:22:09.3251306Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-04-03T16:22:09.3252590Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-04-03T16:22:09.3254045Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-04-03T16:22:09.3255677Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-04-03T16:22:09.3257227Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-04-03T16:22:09.3258713Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-04-03T16:22:09.3260241Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-04-03T16:22:09.3261907Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-04-03T16:22:09.3263461Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-04-03T16:22:09.3264997Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-04-03T16:22:09.3266517Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-04-03T16:22:09.3268126Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-04-03T16:22:09.3417491Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-04-03T16:22:09.3419046Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-04-03T16:22:09.3420569Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-04-03T16:22:09.3422278Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-04-03T16:22:09.3423920Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-04-03T16:22:09.3425488Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-04-03T16:22:09.3427072Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-04-03T16:22:09.3428676Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-04-03T16:22:09.3430284Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-04-03T16:22:09.3431884Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-04-03T16:22:09.3433480Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-04-03T16:22:09.3450054Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-04-03T16:22:09.3518546Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-04-03T16:22:09.3520482Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-04-03T16:22:09.3521961Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-04-03T16:22:09.3523294Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-04-03T16:22:09.3524787Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-04-03T16:22:09.3525995Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-04-03T16:22:09.3527238Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-04-03T16:22:09.3528404Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-04-03T16:22:09.3529451Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-04-03T16:22:09.3530553Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-04-03T16:22:09.3656165Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-04-03T16:22:09.3709292Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-04-03T16:22:09.3710004Z creating: build/lib/ 2024-04-03T16:22:09.3721375Z inflating: build/lib/libpthreadpool.a 2024-04-03T16:22:09.3722162Z inflating: build/lib/libclog.a 2024-04-03T16:22:09.3732299Z inflating: build/lib/libcpuinfo_internals.a 2024-04-03T16:22:09.3742246Z inflating: build/lib/libcpuinfo.a 2024-04-03T16:22:09.3853564Z inflating: build/lib/libprotobuf-lite.a 2024-04-03T16:22:09.3855824Z inflating: build/lib/libnnpack_reference_layers.a 2024-04-03T16:22:09.4423281Z inflating: build/lib/libprotobuf.a 2024-04-03T16:22:09.4504408Z inflating: build/lib/libgtest.a 2024-04-03T16:22:09.4597478Z inflating: build/lib/libbenchmark.a 2024-04-03T16:22:09.4607185Z inflating: build/lib/libittnotify.a 2024-04-03T16:22:09.4641832Z inflating: build/lib/libtensorpipe_uv.a 2024-04-03T16:22:09.4721097Z inflating: build/lib/libasmjit.a 2024-04-03T16:22:09.4879435Z inflating: build/lib/libgloo.a 2024-04-03T16:22:09.4906532Z inflating: build/lib/libfmt.a 2024-04-03T16:22:09.4908790Z inflating: build/lib/libcaffe2_nvrtc.so 2024-04-03T16:22:09.4909592Z inflating: build/lib/libfoxi_loader.a 2024-04-03T16:22:09.5021207Z inflating: build/lib/libc10.so 2024-04-03T16:22:09.5022648Z inflating: build/lib/libtorch_global_deps.so 2024-04-03T16:22:09.5039813Z inflating: build/lib/libqnnpack.a 2024-04-03T16:22:09.5666661Z inflating: build/lib/libprotoc.a 2024-04-03T16:22:09.5691035Z inflating: build/lib/libpytorch_qnnpack.a 2024-04-03T16:22:09.5713380Z inflating: build/lib/libgmock.a 2024-04-03T16:22:09.5714013Z inflating: build/lib/libgtest_main.a 2024-04-03T16:22:09.5715052Z inflating: build/lib/libbenchmark_main.a 2024-04-03T16:22:10.7539542Z inflating: build/lib/libdnnl.a 2024-04-03T16:22:10.7562268Z inflating: build/lib/libnnpack.a 2024-04-03T16:22:10.8276593Z inflating: build/lib/libtensorpipe.a 2024-04-03T16:22:10.8343966Z inflating: build/lib/libc10_cuda.so 2024-04-03T16:22:10.8344559Z inflating: build/lib/libgmock_main.a 2024-04-03T16:22:10.9907162Z inflating: build/lib/libfbgemm.a 2024-04-03T16:22:11.0544376Z inflating: build/lib/libkineto.a 2024-04-03T16:22:11.0862550Z inflating: build/lib/libtensorpipe_cuda.a 2024-04-03T16:22:11.0910441Z inflating: build/lib/libcaffe2_protos.a 2024-04-03T16:22:11.0962399Z inflating: build/lib/libonnx_proto.a 2024-04-03T16:22:11.1192756Z inflating: build/lib/libXNNPACK.a 2024-04-03T16:22:11.1677519Z inflating: build/lib/libgloo_cuda.a 2024-04-03T16:22:11.2521107Z inflating: build/lib/libonnx.a 2024-04-03T16:22:14.1725837Z inflating: build/lib/libtorch_cpu.so 2024-04-03T16:22:14.1730223Z inflating: build/lib/libshm.so 2024-04-03T16:22:14.1735893Z inflating: build/lib/libunbox_lib.a 2024-04-03T16:22:16.6185188Z inflating: build/lib/libtorch_cuda.so 2024-04-03T16:22:16.6186069Z inflating: build/lib/libtorch.so 2024-04-03T16:22:16.6189272Z inflating: build/lib/libc10d_cuda_test.so 2024-04-03T16:22:17.6294504Z inflating: build/lib/libtorch_cuda_linalg.so 2024-04-03T16:22:17.6319920Z inflating: build/lib/libjitbackend_test.so 2024-04-03T16:22:17.6391027Z inflating: build/lib/libtorchbind_test.so 2024-04-03T16:22:17.6422549Z inflating: build/lib/libbackend_with_compiler.so 2024-04-03T16:22:17.8760798Z inflating: build/lib/libtorch_python.so 2024-04-03T16:22:17.8803715Z inflating: build/lib/libnnapi_backend.so 2024-04-03T16:22:17.8804290Z creating: build/bin/ 2024-04-03T16:22:17.8866723Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-04-03T16:22:17.8931051Z inflating: build/bin/c10_DeviceGuard_test 2024-04-03T16:22:17.8993752Z inflating: build/bin/c10_Device_test 2024-04-03T16:22:17.9067650Z inflating: build/bin/c10_DispatchKeySet_test 2024-04-03T16:22:17.9134695Z inflating: build/bin/c10_Scalar_test 2024-04-03T16:22:17.9196504Z inflating: build/bin/c10_StreamGuard_test 2024-04-03T16:22:17.9260191Z inflating: build/bin/c10_SymInt_test 2024-04-03T16:22:17.9328292Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-04-03T16:22:17.9397936Z inflating: build/bin/c10_SizesAndStrides_test 2024-04-03T16:22:17.9467589Z inflating: build/bin/c10_InlineStreamGuard_test 2024-04-03T16:22:17.9533640Z inflating: build/bin/c10_Bitset_test 2024-04-03T16:22:17.9622862Z inflating: build/bin/c10_cow_test 2024-04-03T16:22:17.9683817Z inflating: build/bin/c10_ConstexprCrc_test 2024-04-03T16:22:17.9745960Z inflating: build/bin/c10_DeadlockDetection_test 2024-04-03T16:22:17.9809436Z inflating: build/bin/c10_Half_test 2024-04-03T16:22:17.9879576Z inflating: build/bin/c10_LeftRight_test 2024-04-03T16:22:17.9948338Z inflating: build/bin/c10_Metaprogramming_test 2024-04-03T16:22:18.0010907Z inflating: build/bin/c10_Synchronized_test 2024-04-03T16:22:18.0080594Z inflating: build/bin/c10_ThreadLocal_test 2024-04-03T16:22:18.0144852Z inflating: build/bin/c10_TypeIndex_test 2024-04-03T16:22:18.0208689Z inflating: build/bin/c10_TypeList_test 2024-04-03T16:22:18.0269778Z inflating: build/bin/c10_TypeTraits_test 2024-04-03T16:22:18.0334559Z inflating: build/bin/c10_accumulate_test 2024-04-03T16:22:18.0404027Z inflating: build/bin/c10_bfloat16_test 2024-04-03T16:22:18.0466841Z inflating: build/bin/c10_bit_cast_test 2024-04-03T16:22:18.0537318Z inflating: build/bin/c10_complex_math_test 2024-04-03T16:22:18.0607104Z inflating: build/bin/c10_complex_test 2024-04-03T16:22:18.0672746Z inflating: build/bin/c10_exception_test 2024-04-03T16:22:18.0735747Z inflating: build/bin/c10_flags_test 2024-04-03T16:22:18.0798059Z inflating: build/bin/c10_generic_math_test 2024-04-03T16:22:18.1003995Z inflating: build/bin/c10_intrusive_ptr_test 2024-04-03T16:22:18.1067264Z inflating: build/bin/c10_irange_test 2024-04-03T16:22:18.1136994Z inflating: build/bin/c10_logging_test 2024-04-03T16:22:18.1230726Z inflating: build/bin/c10_optional_test 2024-04-03T16:22:18.1308502Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-04-03T16:22:18.1375861Z inflating: build/bin/c10_registry_test 2024-04-03T16:22:18.1564942Z inflating: build/bin/c10_small_vector_test 2024-04-03T16:22:18.1629258Z inflating: build/bin/c10_ssize_test 2024-04-03T16:22:18.1694460Z inflating: build/bin/c10_string_util_test 2024-04-03T16:22:18.1767839Z inflating: build/bin/c10_string_view_test 2024-04-03T16:22:18.1830629Z inflating: build/bin/c10_tempfile_test 2024-04-03T16:22:18.1900727Z inflating: build/bin/c10_typeid_test 2024-04-03T16:22:18.1961359Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-04-03T16:22:18.2519383Z inflating: build/bin/protoc-3.13.0.0 2024-04-03T16:22:18.3076266Z inflating: build/bin/protoc 2024-04-03T16:22:18.3142557Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-04-03T16:22:18.3208696Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-04-03T16:22:18.3274509Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-04-03T16:22:18.3339756Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-04-03T16:22:18.3406306Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-04-03T16:22:18.3467342Z inflating: build/bin/c10_cuda_CUDATest 2024-04-03T16:22:18.3533265Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-04-03T16:22:18.3599262Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-04-03T16:22:18.4010369Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-04-03T16:22:18.4456447Z inflating: build/bin/vec_test_all_types_AVX2 2024-04-03T16:22:18.4886155Z inflating: build/bin/vec_test_all_types_AVX512 2024-04-03T16:22:18.4952743Z inflating: build/bin/HashStoreTest 2024-04-03T16:22:18.5022325Z inflating: build/bin/TCPStoreTest 2024-04-03T16:22:18.5088583Z inflating: build/bin/FileStoreTest 2024-04-03T16:22:18.5105434Z inflating: build/bin/ProcessGroupMPITest 2024-04-03T16:22:18.5110690Z inflating: build/bin/torch_shm_manager 2024-04-03T16:22:18.5178111Z inflating: build/bin/test_edge_op_registration 2024-04-03T16:22:18.5181618Z inflating: build/bin/example_allreduce 2024-04-03T16:22:18.5246708Z inflating: build/bin/Dimname_test 2024-04-03T16:22:18.5339200Z inflating: build/bin/Dict_test 2024-04-03T16:22:18.5419750Z inflating: build/bin/MaybeOwned_test 2024-04-03T16:22:18.5491521Z inflating: build/bin/NamedTensor_test 2024-04-03T16:22:18.5565771Z inflating: build/bin/apply_utils_test 2024-04-03T16:22:18.5639226Z inflating: build/bin/atest 2024-04-03T16:22:18.5717997Z inflating: build/bin/basic 2024-04-03T16:22:18.5786868Z inflating: build/bin/broadcast_test 2024-04-03T16:22:18.5850373Z inflating: build/bin/cpu_allocator_test 2024-04-03T16:22:18.5923052Z inflating: build/bin/cpu_generator_test 2024-04-03T16:22:18.5989415Z inflating: build/bin/cpu_profiling_allocator_test 2024-04-03T16:22:18.6105750Z inflating: build/bin/cpu_rng_test 2024-04-03T16:22:18.6168444Z inflating: build/bin/dispatch_key_set_test 2024-04-03T16:22:18.6231363Z inflating: build/bin/dlconvertor_test 2024-04-03T16:22:18.6303604Z inflating: build/bin/extension_backend_test 2024-04-03T16:22:18.6372943Z inflating: build/bin/half_test 2024-04-03T16:22:18.6492364Z inflating: build/bin/ivalue_test 2024-04-03T16:22:18.6554480Z inflating: build/bin/lazy_tensor_test 2024-04-03T16:22:18.6622717Z inflating: build/bin/math_kernel_test 2024-04-03T16:22:18.6690357Z inflating: build/bin/memory_format_test 2024-04-03T16:22:18.6756598Z inflating: build/bin/memory_overlapping_test 2024-04-03T16:22:18.6820068Z inflating: build/bin/operator_name_test 2024-04-03T16:22:18.6886271Z inflating: build/bin/mobile_memory_cleanup 2024-04-03T16:22:18.6956167Z inflating: build/bin/native_test 2024-04-03T16:22:18.7020642Z inflating: build/bin/operators_test 2024-04-03T16:22:18.7092273Z inflating: build/bin/quantized_test 2024-04-03T16:22:18.7157206Z inflating: build/bin/packedtensoraccessor_test 2024-04-03T16:22:18.7241733Z inflating: build/bin/pow_test 2024-04-03T16:22:18.7303767Z inflating: build/bin/reduce_ops_test 2024-04-03T16:22:18.7367950Z inflating: build/bin/reportMemoryUsage_test 2024-04-03T16:22:18.7438554Z inflating: build/bin/scalar_tensor_test 2024-04-03T16:22:18.7511855Z inflating: build/bin/scalar_test 2024-04-03T16:22:18.7576670Z inflating: build/bin/StorageUtils_test 2024-04-03T16:22:18.7642581Z inflating: build/bin/stride_properties_test 2024-04-03T16:22:18.7741139Z inflating: build/bin/tensor_iterator_test 2024-04-03T16:22:18.7808785Z inflating: build/bin/test_parallel 2024-04-03T16:22:18.7812695Z inflating: build/bin/thread_init_test 2024-04-03T16:22:18.7882423Z inflating: build/bin/type_ptr_test 2024-04-03T16:22:18.7957266Z inflating: build/bin/type_test 2024-04-03T16:22:18.8022410Z inflating: build/bin/undefined_tensor_test 2024-04-03T16:22:18.8024112Z inflating: build/bin/verify_api_visibility 2024-04-03T16:22:18.8111735Z inflating: build/bin/legacy_vmap_test 2024-04-03T16:22:18.8176156Z inflating: build/bin/weakref_test 2024-04-03T16:22:18.8240726Z inflating: build/bin/wrapdim_test 2024-04-03T16:22:18.8305968Z inflating: build/bin/xla_tensor_test 2024-04-03T16:22:18.8381715Z inflating: build/bin/IListRef_test 2024-04-03T16:22:18.8514320Z inflating: build/bin/List_test 2024-04-03T16:22:18.8666863Z inflating: build/bin/kernel_function_legacy_test 2024-04-03T16:22:18.8750175Z inflating: build/bin/KernelFunction_test 2024-04-03T16:22:18.8870926Z inflating: build/bin/kernel_function_test 2024-04-03T16:22:18.9031786Z inflating: build/bin/kernel_lambda_legacy_test 2024-04-03T16:22:18.9161606Z inflating: build/bin/kernel_lambda_test 2024-04-03T16:22:18.9237778Z inflating: build/bin/kernel_stackbased_test 2024-04-03T16:22:18.9301464Z inflating: build/bin/CppSignature_test 2024-04-03T16:22:18.9422062Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-04-03T16:22:18.9483238Z inflating: build/bin/op_allowlist_test 2024-04-03T16:22:18.9552939Z inflating: build/bin/backend_fallback_test 2024-04-03T16:22:18.9930511Z inflating: build/bin/op_registration_test 2024-04-03T16:22:19.0009908Z inflating: build/bin/inline_container_test 2024-04-03T16:22:19.0076238Z inflating: build/bin/cuda_apply_test 2024-04-03T16:22:19.0139786Z inflating: build/bin/cuda_allocator_test 2024-04-03T16:22:19.0208064Z inflating: build/bin/cuda_caching_host_allocator_test 2024-04-03T16:22:19.0281886Z inflating: build/bin/cuda_atomic_ops_test 2024-04-03T16:22:19.0369853Z inflating: build/bin/cuda_complex_math_test 2024-04-03T16:22:19.0431996Z inflating: build/bin/cuda_device_test 2024-04-03T16:22:19.0505748Z inflating: build/bin/cuda_complex_test 2024-04-03T16:22:19.0577622Z inflating: build/bin/cuda_cub_test 2024-04-03T16:22:19.0641268Z inflating: build/bin/cuda_dlconvertor_test 2024-04-03T16:22:19.0705064Z inflating: build/bin/cuda_integer_divider_test 2024-04-03T16:22:19.0786899Z inflating: build/bin/cuda_distributions_test 2024-04-03T16:22:19.0858031Z inflating: build/bin/cuda_generator_test 2024-04-03T16:22:19.0919979Z inflating: build/bin/cuda_half_test 2024-04-03T16:22:19.0981543Z inflating: build/bin/cuda_optional_test 2024-04-03T16:22:19.1044122Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-04-03T16:22:19.1109643Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-04-03T16:22:19.1185552Z inflating: build/bin/cuda_stream_test 2024-04-03T16:22:19.1250158Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-04-03T16:22:19.1311741Z inflating: build/bin/cuda_cudnn_test 2024-04-03T16:22:19.1376954Z inflating: build/bin/cuda_vectorized_test 2024-04-03T16:22:19.1395681Z inflating: build/bin/tutorial_tensorexpr 2024-04-03T16:22:19.1477647Z inflating: build/bin/ProcessGroupGlooTest 2024-04-03T16:22:19.1549167Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-04-03T16:22:19.1627502Z inflating: build/bin/ProcessGroupNCCLTest 2024-04-03T16:22:19.1704717Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-04-03T16:22:19.1773459Z inflating: build/bin/test_dist_autograd 2024-04-03T16:22:19.1776583Z inflating: build/bin/parallel_benchmark 2024-04-03T16:22:19.1863112Z inflating: build/bin/test_cpp_rpc 2024-04-03T16:22:19.1947043Z inflating: build/bin/test_mobile_nnc 2024-04-03T16:22:19.1957967Z inflating: build/bin/aot_model_compiler_test 2024-04-03T16:22:19.2390599Z inflating: build/bin/test_lazy 2024-04-03T16:22:19.3430861Z inflating: build/bin/test_tensorexpr 2024-04-03T16:22:19.4137924Z inflating: build/bin/test_jit 2024-04-03T16:22:19.5613721Z inflating: build/bin/test_api 2024-04-03T16:22:19.5614299Z creating: .additional_ci_files/ 2024-04-03T16:22:19.5658888Z inflating: .additional_ci_files/test-times.json 2024-04-03T16:22:19.5838911Z inflating: .additional_ci_files/test-class-times.json 2024-04-03T16:22:19.5878447Z ##[group]Run rm artifacts.zip 2024-04-03T16:22:19.5878945Z rm artifacts.zip 2024-04-03T16:22:19.5888239Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:22:19.5888888Z env: 2024-04-03T16:22:19.5889226Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:19.5889772Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:19.5890380Z ##[endgroup] 2024-04-03T16:22:19.7040079Z ##[group]Run df -H 2024-04-03T16:22:19.7040475Z df -H 2024-04-03T16:22:19.7048996Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:22:19.7049616Z env: 2024-04-03T16:22:19.7049940Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:19.7050508Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:19.7051113Z ##[endgroup] 2024-04-03T16:22:19.7086243Z Filesystem Size Used Avail Use% Mounted on 2024-04-03T16:22:19.7086902Z devtmpfs 65G 0 65G 0% /dev 2024-04-03T16:22:19.7087511Z tmpfs 65G 0 65G 0% /dev/shm 2024-04-03T16:22:19.7088380Z tmpfs 65G 463k 65G 1% /run 2024-04-03T16:22:19.7088961Z tmpfs 65G 0 65G 0% /sys/fs/cgroup 2024-04-03T16:22:19.7089754Z /dev/xvda1 162G 36G 126G 22% / 2024-04-03T16:22:19.7162872Z Prepare all required actions 2024-04-03T16:22:19.7163506Z Getting action download info 2024-04-03T16:22:19.9019803Z ##[group]Run ./.github/actions/download-td-artifacts 2024-04-03T16:22:19.9020385Z with: 2024-04-03T16:22:19.9020671Z env: 2024-04-03T16:22:19.9020999Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:19.9021540Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:19.9022144Z ##[endgroup] 2024-04-03T16:22:19.9065373Z ##[group]Run seemethere/download-artifact-s3@v4 2024-04-03T16:22:19.9065932Z with: 2024-04-03T16:22:19.9066235Z name: td_results 2024-04-03T16:22:19.9066626Z s3-bucket: gha-artifacts 2024-04-03T16:22:19.9067056Z region: us-east-1 2024-04-03T16:22:19.9067415Z env: 2024-04-03T16:22:19.9067739Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:19.9068293Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:19.9068875Z ##[endgroup] 2024-04-03T16:22:20.5203608Z (node:128873) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-04-03T16:22:20.5204377Z 2024-04-03T16:22:20.5204711Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-04-03T16:22:20.5205735Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-04-03T16:22:20.5206819Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-04-03T16:22:20.6090327Z Found 1 objects with prefix pytorch/pytorch/8536950919/td_results/ 2024-04-03T16:22:20.6091743Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-04-03T16:22:20.6902270Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-04-03T16:22:20.6913946Z Artifact download has finished successfully 2024-04-03T16:22:20.7108816Z ##[group]Run mkdir -p .additional_ci_files 2024-04-03T16:22:20.7109418Z mkdir -p .additional_ci_files 2024-04-03T16:22:20.7110139Z mv td_results.json .additional_ci_files/td_results.json 2024-04-03T16:22:20.7119318Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:22:20.7119974Z env: 2024-04-03T16:22:20.7120313Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:20.7120856Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:20.7121463Z ##[endgroup] 2024-04-03T16:22:20.7230456Z ##[group]Run .github/scripts/parse_ref.py 2024-04-03T16:22:20.7231109Z .github/scripts/parse_ref.py 2024-04-03T16:22:20.7240078Z shell: /usr/bin/bash -e {0} 2024-04-03T16:22:20.7240509Z env: 2024-04-03T16:22:20.7240851Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:20.7241419Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:20.7242015Z ##[endgroup] 2024-04-03T16:22:20.7555265Z Prepare all required actions 2024-04-03T16:22:20.7607493Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-04-03T16:22:20.7608063Z with: 2024-04-03T16:22:20.7608776Z github-token: *** 2024-04-03T16:22:20.7609142Z env: 2024-04-03T16:22:20.7609476Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:20.7610032Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:20.7610617Z ##[endgroup] 2024-04-03T16:22:20.7633326Z ##[group]Run set -eux 2024-04-03T16:22:20.7633743Z set -eux 2024-04-03T16:22:20.7634482Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-04-03T16:22:20.7643135Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:22:20.7643737Z env: 2024-04-03T16:22:20.7644071Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:20.7644630Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:20.7645518Z GITHUB_TOKEN: *** 2024-04-03T16:22:20.7646029Z ##[endgroup] 2024-04-03T16:22:20.7672030Z + python3 .github/scripts/get_workflow_job_id.py 8536950919 i-044537176b517f11b 2024-04-03T16:22:29.2161033Z setting job-id=23402695647 2024-04-03T16:22:29.2162103Z setting job-name=linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) 2024-04-03T16:22:29.2413830Z Prepare all required actions 2024-04-03T16:22:29.2414457Z Getting action download info 2024-04-03T16:22:29.3488978Z ##[group]Run ./.github/actions/filter-test-configs 2024-04-03T16:22:29.3489553Z with: 2024-04-03T16:22:29.3490123Z github-token: *** 2024-04-03T16:22:29.3493210Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "deploy", "shard": 1, "num_shards": 1, "runner": "linux.4xlarge.nvidia.gpu"}]} 2024-04-03T16:22:29.3496860Z job-name: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) 2024-04-03T16:22:29.3497780Z env: 2024-04-03T16:22:29.3498103Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:29.3498666Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:29.3499269Z ##[endgroup] 2024-04-03T16:22:29.3554173Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-04-03T16:22:29.3554874Z with: 2024-04-03T16:22:29.3555194Z shell: bash 2024-04-03T16:22:29.3555542Z timeout_minutes: 10 2024-04-03T16:22:29.3555941Z max_attempts: 5 2024-04-03T16:22:29.3556324Z retry_wait_seconds: 30 2024-04-03T16:22:29.3557279Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore python3 -m pip install requests==2.26.0 pyyaml==6.0.1 2024-04-03T16:22:29.3558335Z polling_interval_seconds: 1 2024-04-03T16:22:29.3558807Z warning_on_retry: true 2024-04-03T16:22:29.3559551Z continue_on_error: false 2024-04-03T16:22:29.3559968Z env: 2024-04-03T16:22:29.3560297Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:29.3560836Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:29.3561662Z GITHUB_TOKEN: *** 2024-04-03T16:22:29.3562057Z ##[endgroup] 2024-04-03T16:22:29.4282258Z + python3 -m pip install requests==2.26.0 pyyaml==6.0.1 2024-04-03T16:22:29.7176043Z Defaulting to user installation because normal site-packages is not writeable 2024-04-03T16:22:29.7386675Z Requirement already satisfied: requests==2.26.0 in /home/ec2-user/.local/lib/python3.7/site-packages (2.26.0) 2024-04-03T16:22:29.7570533Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-04-03T16:22:29.7581958Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.26.0) (3.6) 2024-04-03T16:22:29.7597212Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.26.0) (2024.2.2) 2024-04-03T16:22:29.7608253Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.26.0) (1.26.18) 2024-04-03T16:22:29.7851613Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.26.0) (2.0.12) 2024-04-03T16:22:30.4275230Z Command completed after 1 attempt(s). 2024-04-03T16:22:30.4329596Z ##[group]Run set -x 2024-04-03T16:22:30.4330007Z set -x 2024-04-03T16:22:30.4330367Z  2024-04-03T16:22:30.4331040Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-04-03T16:22:30.4331877Z # in runner workspace 2024-04-03T16:22:30.4332534Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-04-03T16:22:30.4341466Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:22:30.4342092Z env: 2024-04-03T16:22:30.4342430Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:30.4342970Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:30.4343764Z ##[endgroup] 2024-04-03T16:22:30.4409593Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-04-03T16:22:30.4681785Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-04-03T16:22:30.4682451Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-04-03T16:22:30.4683027Z echo "Job name: ${JOB_NAME}" 2024-04-03T16:22:30.4683515Z  2024-04-03T16:22:30.4684168Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-04-03T16:22:30.4685018Z # in runner workspace 2024-04-03T16:22:30.4685735Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-04-03T16:22:30.4686559Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-04-03T16:22:30.4687105Z  --job-name "${JOB_NAME}" \ 2024-04-03T16:22:30.4690410Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "deploy", "shard": 1, "num_shards": 1, "runner": "linux.4xlarge.nvidia.gpu"}]}" \ 2024-04-03T16:22:30.4693771Z  --pr-number "${PR_NUMBER}" \ 2024-04-03T16:22:30.4694297Z  --tag "${TAG}" \ 2024-04-03T16:22:30.4694758Z  --event-name "${EVENT_NAME}" \ 2024-04-03T16:22:30.4695302Z  --schedule "${SCHEDULE}" \ 2024-04-03T16:22:30.4695836Z  --branch "${HEAD_BRANCH}" 2024-04-03T16:22:30.4703965Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:22:30.4704568Z env: 2024-04-03T16:22:30.4704901Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:30.4705468Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:30.4706314Z GITHUB_TOKEN: *** 2024-04-03T16:22:30.4707077Z JOB_NAME: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) 2024-04-03T16:22:30.4707961Z PR_NUMBER: 123154 2024-04-03T16:22:30.4708329Z TAG: 2024-04-03T16:22:30.4708648Z EVENT_NAME: pull_request 2024-04-03T16:22:30.4709073Z SCHEDULE: 2024-04-03T16:22:30.4709418Z HEAD_BRANCH: 2024-04-03T16:22:30.4709762Z ##[endgroup] 2024-04-03T16:22:30.4735619Z Workflow: pull 2024-04-03T16:22:30.4737114Z Job name: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) 2024-04-03T16:22:30.7001071Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-04-03T16:22:30.7882290Z fatal: unknown commit main 2024-04-03T16:22:30.7909988Z /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/filter_test_configs.py:458: UserWarning: failed to get commit messages: Command '['git', 'cherry', '-v', 'main']' returned non-zero exit status 128. 2024-04-03T16:22:30.7912038Z warnings.warn(f"failed to get commit messages: {e}") 2024-04-03T16:22:30.8057778Z ##[group]Run echo "Filtered matrix:" 2024-04-03T16:22:30.8058333Z echo "Filtered matrix:" 2024-04-03T16:22:30.8061537Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "linux.4xlarge.nvidia.gpu"}, {"config": "deploy", "shard": 1, "num_shards": 1, "runner": "linux.4xlarge.nvidia.gpu"}]}" 2024-04-03T16:22:30.8064811Z  2024-04-03T16:22:30.8065123Z echo 2024-04-03T16:22:30.8065771Z echo "Is the current job unstable? False" 2024-04-03T16:22:30.8066371Z  2024-04-03T16:22:30.8066680Z echo 2024-04-03T16:22:30.8067123Z echo "Is keep-going label set? False" 2024-04-03T16:22:30.8067665Z  2024-04-03T16:22:30.8067971Z echo 2024-04-03T16:22:30.8068356Z echo "Renabled issues? " 2024-04-03T16:22:30.8076890Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:22:30.8077509Z env: 2024-04-03T16:22:30.8077828Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:30.8078388Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:30.8078989Z ##[endgroup] 2024-04-03T16:22:30.8104437Z Filtered matrix: 2024-04-03T16:22:30.8107751Z {include: [{config: default, shard: 1, num_shards: 5, runner: linux.4xlarge.nvidia.gpu}, {config: default, shard: 2, num_shards: 5, runner: linux.4xlarge.nvidia.gpu}, {config: default, shard: 3, num_shards: 5, runner: linux.4xlarge.nvidia.gpu}, {config: default, shard: 4, num_shards: 5, runner: linux.4xlarge.nvidia.gpu}, {config: default, shard: 5, num_shards: 5, runner: linux.4xlarge.nvidia.gpu}, {config: deploy, shard: 1, num_shards: 1, runner: linux.4xlarge.nvidia.gpu}]} 2024-04-03T16:22:30.8110714Z 2024-04-03T16:22:30.8110906Z Is the current job unstable? False 2024-04-03T16:22:30.8111258Z 2024-04-03T16:22:30.8111538Z Is keep-going label set? False 2024-04-03T16:22:30.8111859Z 2024-04-03T16:22:30.8112007Z Renabled issues? 2024-04-03T16:22:30.8165765Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-04-03T16:22:30.8166691Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-04-03T16:22:30.8174600Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:22:30.8175229Z env: 2024-04-03T16:22:30.8175546Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:30.8176102Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:30.8176708Z JOB_TIMEOUT: 360 2024-04-03T16:22:30.8177069Z ##[endgroup] 2024-04-03T16:22:30.8271201Z ##[group]Run set -x 2024-04-03T16:22:30.8271728Z set -x 2024-04-03T16:22:30.8272091Z  2024-04-03T16:22:30.8272523Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-04-03T16:22:30.8273188Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-04-03T16:22:30.8273895Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-04-03T16:22:30.8274529Z  TEST_COMMAND=.ci/onnx/test.sh 2024-04-03T16:22:30.8275020Z else 2024-04-03T16:22:30.8275439Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-04-03T16:22:30.8275958Z fi 2024-04-03T16:22:30.8276275Z  2024-04-03T16:22:30.8276849Z # detached container should get cleaned up by teardown_ec2_linux 2024-04-03T16:22:30.8278028Z # TODO: Stop building test binaries as part of the build phase 2024-04-03T16:22:30.8278862Z # Used for GPU_FLAG since that doesn't play nice 2024-04-03T16:22:30.8280060Z # shellcheck disable=SC2086,SC2090 2024-04-03T16:22:30.8280632Z container_name=$(docker run \ 2024-04-03T16:22:30.8281153Z  ${GPU_FLAG:-} \ 2024-04-03T16:22:30.8281610Z  -e BUILD_ENVIRONMENT \ 2024-04-03T16:22:30.8282086Z  -e PR_NUMBER \ 2024-04-03T16:22:30.8282530Z  -e GITHUB_ACTIONS \ 2024-04-03T16:22:30.8283013Z  -e GITHUB_REPOSITORY \ 2024-04-03T16:22:30.8283494Z  -e GITHUB_WORKFLOW \ 2024-04-03T16:22:30.8283971Z  -e GITHUB_JOB \ 2024-04-03T16:22:30.8284411Z  -e GITHUB_RUN_ID \ 2024-04-03T16:22:30.8284882Z  -e GITHUB_RUN_NUMBER \ 2024-04-03T16:22:30.8285362Z  -e GITHUB_RUN_ATTEMPT \ 2024-04-03T16:22:30.8285850Z  -e JOB_ID \ 2024-04-03T16:22:30.8286279Z  -e JOB_NAME \ 2024-04-03T16:22:30.8286689Z  -e BASE_SHA \ 2024-04-03T16:22:30.8287110Z  -e BRANCH \ 2024-04-03T16:22:30.8287521Z  -e SHA1 \ 2024-04-03T16:22:30.8287929Z  -e AWS_DEFAULT_REGION \ 2024-04-03T16:22:30.8288430Z  -e IN_WHEEL_TEST \ 2024-04-03T16:22:30.8288892Z  -e SHARD_NUMBER \ 2024-04-03T16:22:30.8289327Z  -e TEST_CONFIG \ 2024-04-03T16:22:30.8289778Z  -e NUM_TEST_SHARDS \ 2024-04-03T16:22:30.8290261Z  -e REENABLED_ISSUES \ 2024-04-03T16:22:30.8290750Z  -e CONTINUE_THROUGH_ERROR \ 2024-04-03T16:22:30.8291271Z  -e VERBOSE_TEST_LOGS \ 2024-04-03T16:22:30.8291763Z  -e NO_TEST_TIMEOUT \ 2024-04-03T16:22:30.8292211Z  -e NO_TD \ 2024-04-03T16:22:30.8292621Z  -e PR_LABELS \ 2024-04-03T16:22:30.8293121Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-04-03T16:22:30.8293711Z  -e SCCACHE_BUCKET \ 2024-04-03T16:22:30.8294214Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-04-03T16:22:30.8294717Z  -e XLA_CUDA \ 2024-04-03T16:22:30.8295192Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-04-03T16:22:30.8295829Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-04-03T16:22:30.8296485Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-04-03T16:22:30.8297106Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-04-03T16:22:30.8297683Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-04-03T16:22:30.8298198Z  -e DASHBOARD_TAG \ 2024-04-03T16:22:30.8298771Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-04-03T16:22:30.8299439Z  --ulimit stack=10485760:83886080 \ 2024-04-03T16:22:30.8300039Z  --security-opt seccomp=unconfined \ 2024-04-03T16:22:30.8300615Z  --cap-add=SYS_PTRACE \ 2024-04-03T16:22:30.8301078Z  --ipc=host \ 2024-04-03T16:22:30.8301513Z  --shm-size="${SHM_SIZE}" \ 2024-04-03T16:22:30.8302009Z  --tty \ 2024-04-03T16:22:30.8302375Z  --detach \ 2024-04-03T16:22:30.8302807Z  --name="${container_name}" \ 2024-04-03T16:22:30.8303322Z  --user jenkins \ 2024-04-03T16:22:30.8304064Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-04-03T16:22:30.8304792Z  -w /var/lib/jenkins/workspace \ 2024-04-03T16:22:30.8305331Z  "${DOCKER_IMAGE}" 2024-04-03T16:22:30.8305748Z ) 2024-04-03T16:22:30.8306251Z # Propagate download.pytorch.org IP to container 2024-04-03T16:22:30.8307417Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-04-03T16:22:30.8308617Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-04-03T16:22:30.8309773Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-04-03T16:22:30.8318177Z shell: /usr/bin/bash -e {0} 2024-04-03T16:22:30.8318760Z env: 2024-04-03T16:22:30.8319387Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:22:30.8320045Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:30.8321266Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9 2024-04-03T16:22:30.8321875Z PR_NUMBER: 123154 2024-04-03T16:22:30.8322293Z GITHUB_REPOSITORY: pytorch/pytorch 2024-04-03T16:22:30.8322783Z GITHUB_WORKFLOW: pull 2024-04-03T16:22:30.8323189Z GITHUB_JOB: test 2024-04-03T16:22:30.8323618Z GITHUB_RUN_ID: 8536950919 2024-04-03T16:22:30.8324124Z GITHUB_RUN_NUMBER: 192590 2024-04-03T16:22:30.8324462Z GITHUB_RUN_ATTEMPT: 2 2024-04-03T16:22:30.8324844Z JOB_ID: 23402695647 2024-04-03T16:22:30.8325618Z JOB_NAME: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) 2024-04-03T16:22:30.8326625Z BRANCH: pull/123154 2024-04-03T16:22:30.8327031Z SHA1: d253bd7befc64137de913dbb8727b511f1052411 2024-04-03T16:22:30.8327688Z BASE_SHA: 1d52c2d98507759a2143d52b922c541dbe36b6bf 2024-04-03T16:22:30.8328388Z TEST_CONFIG: default 2024-04-03T16:22:30.8328760Z SHARD_NUMBER: 3 2024-04-03T16:22:30.8329157Z NUM_TEST_SHARDS: 5 2024-04-03T16:22:30.8329541Z REENABLED_ISSUES: 2024-04-03T16:22:30.8329966Z CONTINUE_THROUGH_ERROR: False 2024-04-03T16:22:30.8330445Z VERBOSE_TEST_LOGS: False 2024-04-03T16:22:30.8330961Z NO_TEST_TIMEOUT: False 2024-04-03T16:22:30.8331355Z NO_TD: False 2024-04-03T16:22:30.8331822Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-04-03T16:22:30.8332430Z SCCACHE_S3_KEY_PREFIX: pull 2024-04-03T16:22:30.8332855Z SHM_SIZE: 2g 2024-04-03T16:22:30.8334060Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:22:30.8335363Z XLA_CUDA: 2024-04-03T16:22:30.8335955Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-04-03T16:22:30.8336796Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-04-03T16:22:30.8337256Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-04-03T16:22:30.8337760Z DASHBOARD_TAG: 2024-04-03T16:22:30.8338125Z HUGGING_FACE_HUB_TOKEN: 2024-04-03T16:22:30.8338544Z ##[endgroup] 2024-04-03T16:22:30.8364275Z + [[ default == \m\u\l\t\i\g\p\u ]] 2024-04-03T16:22:30.8364883Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *onnx* ]] 2024-04-03T16:22:30.8365497Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-04-03T16:22:30.8374610Z +++ nproc --ignore=2 2024-04-03T16:22:30.8392387Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_8536950919 --ulimit stack=10485760:83886080 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:22:41.6614306Z + container_name=5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:22:41.6616176Z + grep download.pytorch.org /etc/hosts 2024-04-03T16:22:41.6618350Z + docker exec -i 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 sudo bash -c '/bin/cat >> /etc/hosts' 2024-04-03T16:22:41.7549537Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-04-03T16:22:41.7607595Z + echo DOCKER_CONTAINER_ID=5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:22:41.7611794Z ++ echo dist/torch-2.4.0a0+gitd253bd7-cp310-cp310-linux_x86_64.whl 2024-04-03T16:22:41.7614420Z + docker exec -t 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 sh -c 'pip install dist/torch-2.4.0a0+gitd253bd7-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-04-03T16:22:42.3248560Z Processing ./dist/torch-2.4.0a0+gitd253bd7-cp310-cp310-linux_x86_64.whl 2024-04-03T16:22:43.4239554Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.4.0a0+gitd253bd7) (3.13.1) 2024-04-03T16:22:43.4250845Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.4.0a0+gitd253bd7) (4.10.0) 2024-04-03T16:22:43.4256235Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.4.0a0+gitd253bd7) (1.12) 2024-04-03T16:22:43.4262167Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.4.0a0+gitd253bd7) (2.8.8) 2024-04-03T16:22:43.4268177Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.4.0a0+gitd253bd7) (3.1.3) 2024-04-03T16:22:43.4274076Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.4.0a0+gitd253bd7) (2024.2.0) 2024-04-03T16:22:43.4290240Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.4.0a0+gitd253bd7) (3.3.0) 2024-04-03T16:22:43.4380147Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.4.0a0+gitd253bd7) (1.21.2) 2024-04-03T16:22:43.5039121Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.4.0a0+gitd253bd7) (2.1.5) 2024-04-03T16:22:43.5272935Z Requirement already satisfied: mpmath>=0.19 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.4.0a0+gitd253bd7) (1.3.0) 2024-04-03T16:22:44.6112028Z Installing collected packages: torch 2024-04-03T16:22:56.9524806Z Successfully installed torch-2.4.0a0+gitd253bd7 2024-04-03T16:22:57.0211216Z + echo 'Environment variables:' 2024-04-03T16:22:57.0211795Z Environment variables: 2024-04-03T16:22:57.0214448Z + env 2024-04-03T16:22:57.0219640Z INSTALLED_DB=yes 2024-04-03T16:22:57.0220632Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-04-03T16:22:57.0221478Z NVIDIA_VISIBLE_DEVICES=all 2024-04-03T16:22:57.0222058Z NV_NVML_DEV_VERSION=12.1.105-1 2024-04-03T16:22:57.0222713Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-04-03T16:22:57.0224067Z CONTINUE_THROUGH_ERROR=False 2024-04-03T16:22:57.0224902Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-04-03T16:22:57.0226533Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-04-03T16:22:57.0227394Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9 2024-04-03T16:22:57.0227992Z HOSTNAME=5f8b51bc22a6 2024-04-03T16:22:57.0229734Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0231209Z GITHUB_ACTION=__self 2024-04-03T16:22:57.0231506Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-04-03T16:22:57.0238094Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-04-03T16:22:57.0243827Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-04-03T16:22:57.0244593Z NV_NVTX_VERSION=12.1.105-1 2024-04-03T16:22:57.0244927Z GITHUB_RUN_NUMBER=192590 2024-04-03T16:22:57.0245328Z TEST_CONFIG=default 2024-04-03T16:22:57.0245878Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-04-03T16:22:57.0246327Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-04-03T16:22:57.0247075Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-04-03T16:22:57.0247498Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-04-03T16:22:57.0248032Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-04-03T16:22:57.0248506Z GITHUB_TRIGGERING_ACTOR=soulitzer 2024-04-03T16:22:57.0249054Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-04-03T16:22:57.0249643Z GITHUB_REF_TYPE=branch 2024-04-03T16:22:57.0250060Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-04-03T16:22:57.0250655Z NCCL_VERSION=2.17.1-1 2024-04-03T16:22:57.0251006Z BASE_SHA=1d52c2d98507759a2143d52b922c541dbe36b6bf 2024-04-03T16:22:57.0251549Z XLA_CUDA= 2024-04-03T16:22:57.0251909Z HUGGING_FACE_HUB_TOKEN= 2024-04-03T16:22:57.0254436Z *** 2024-04-03T16:22:57.0254721Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-04-03T16:22:57.0255197Z GITHUB_REPOSITORY_ID=65600975 2024-04-03T16:22:57.0255651Z GITHUB_ACTIONS=true 2024-04-03T16:22:57.0256059Z NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:57.0256812Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-04-03T16:22:57.0257356Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-04-03T16:22:57.0258022Z SHA1=d253bd7befc64137de913dbb8727b511f1052411 2024-04-03T16:22:57.0258571Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-04-03T16:22:57.0259220Z GITHUB_SHA=4302cb7ecb2b1ae758f931ad18aeedf6ae927cb0 2024-04-03T16:22:57.0260094Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/123154/merge 2024-04-03T16:22:57.0260879Z UCC_HOME=/usr 2024-04-03T16:22:57.0261308Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-04-03T16:22:57.0261802Z VERBOSE_TEST_LOGS=False 2024-04-03T16:22:57.0262204Z NVIDIA_PRODUCT_NAME=CUDA 2024-04-03T16:22:57.0262818Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-04-03T16:22:57.0263374Z GITHUB_REF=refs/pull/123154/merge 2024-04-03T16:22:57.0264015Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-04-03T16:22:57.0264359Z SHARD_NUMBER=3 2024-04-03T16:22:57.0264733Z GITHUB_REF_PROTECTED=false 2024-04-03T16:22:57.0265217Z HOME=/var/lib/jenkins 2024-04-03T16:22:57.0265599Z GITHUB_API_URL=https://api.github.com 2024-04-03T16:22:57.0266278Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-04-03T16:22:57.0266709Z UCX_COMMIT=00bcc6bb18fc282eb160623b4c0d300147f579af 2024-04-03T16:22:57.0267302Z SCCACHE_S3_KEY_PREFIX=pull 2024-04-03T16:22:57.0267728Z CUDA_VERSION=12.1.1 2024-04-03T16:22:57.0268232Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-04-03T16:22:57.0268810Z NUM_TEST_SHARDS=5 2024-04-03T16:22:57.0269140Z UCX_HOME=/usr 2024-04-03T16:22:57.0269875Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-04-03T16:22:57.0271205Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0272909Z JOB_NAME=linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) 2024-04-03T16:22:57.0274493Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0275907Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-04-03T16:22:57.0276753Z GITHUB_EVENT_NAME=pull_request 2024-04-03T16:22:57.0277203Z DASHBOARD_TAG= 2024-04-03T16:22:57.0277551Z GITHUB_RUN_ID=8536950919 2024-04-03T16:22:57.0278119Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-04-03T16:22:57.0278868Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-04-03T16:22:57.0280303Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0281598Z GITHUB_ACTOR=caic99 2024-04-03T16:22:57.0282047Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-04-03T16:22:57.0282505Z PR_NUMBER=123154 2024-04-03T16:22:57.0282892Z GITHUB_RUN_ATTEMPT=2 2024-04-03T16:22:57.0283307Z ANACONDA_PYTHON_VERSION=3.10 2024-04-03T16:22:57.0283838Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-04-03T16:22:57.0284411Z TERM=xterm 2024-04-03T16:22:57.0284832Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-04-03T16:22:57.0285330Z INSTALLED_VISION=yes 2024-04-03T16:22:57.0285868Z BRANCH=pull/123154 2024-04-03T16:22:57.0286128Z OPENSSL_ROOT_DIR=/opt/openssl 2024-04-03T16:22:57.0286606Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-04-03T16:22:57.0287139Z CUDA_PATH=/usr/local/cuda 2024-04-03T16:22:57.0288092Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-04-03T16:22:57.0289024Z GITHUB_SERVER_URL=https://github.com 2024-04-03T16:22:57.0289648Z UCC_COMMIT=7cb07a76ccedad7e56ceb136b865eb9319c258ea 2024-04-03T16:22:57.0290232Z REENABLED_ISSUES= 2024-04-03T16:22:57.0290577Z SHLVL=1 2024-04-03T16:22:57.0290893Z MAX_JOBS=14 2024-04-03T16:22:57.0291279Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-04-03T16:22:57.0291705Z NVARCH=x86_64 2024-04-03T16:22:57.0292068Z GITHUB_ACTOR_ID=78061359 2024-04-03T16:22:57.0292638Z GITHUB_WORKFLOW_SHA=4302cb7ecb2b1ae758f931ad18aeedf6ae927cb0 2024-04-03T16:22:57.0293270Z GITHUB_REF_NAME=123154/merge 2024-04-03T16:22:57.0293807Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-04-03T16:22:57.0294636Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-04-03T16:22:57.0295337Z GITHUB_JOB=test 2024-04-03T16:22:57.0295881Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-04-03T16:22:57.0296574Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-04-03T16:22:57.0297197Z NO_TEST_TIMEOUT=False 2024-04-03T16:22:57.0297666Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-04-03T16:22:57.0298223Z GITHUB_REPOSITORY=pytorch/pytorch 2024-04-03T16:22:57.0298727Z NV_NVPROF_VERSION=12.1.105-1 2024-04-03T16:22:57.0299176Z GITHUB_RETENTION_DAYS=90 2024-04-03T16:22:57.0299600Z OPENSSL_DIR=/opt/openssl 2024-04-03T16:22:57.0300011Z GITHUB_ACTION_REPOSITORY= 2024-04-03T16:22:57.0301344Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-04-03T16:22:57.0302718Z GITHUB_BASE_REF=main 2024-04-03T16:22:57.0303117Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-04-03T16:22:57.0303565Z CI=true 2024-04-03T16:22:57.0303950Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-04-03T16:22:57.0304449Z GITHUB_REPOSITORY_OWNER=pytorch 2024-04-03T16:22:57.0304907Z JOB_ID=23402695647 2024-04-03T16:22:57.0305287Z INSTALLED_PROTOBUF=yes 2024-04-03T16:22:57.0305710Z GITHUB_HEAD_REF=patch-1 2024-04-03T16:22:57.0306116Z GITHUB_ACTION_REF= 2024-04-03T16:22:57.0306648Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-04-03T16:22:57.0307226Z GITHUB_WORKFLOW=pull 2024-04-03T16:22:57.0307643Z DEBIAN_FRONTEND=noninteractive 2024-04-03T16:22:57.0308804Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0309987Z NO_TD=False 2024-04-03T16:22:57.0310364Z SKIP_SCCACHE_INITIALIZATION=1 2024-04-03T16:22:57.0310815Z _=/usr/bin/env 2024-04-03T16:22:57.0311397Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-04-03T16:22:57.0440138Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-04-03T16:22:57.0442030Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-04-03T16:22:57.0443425Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-04-03T16:22:57.0445625Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-04-03T16:22:57.0447186Z + BUILD_DIR=build 2024-04-03T16:22:57.0447907Z + BUILD_RENAMED_DIR=build_renamed 2024-04-03T16:22:57.0449205Z + BUILD_BIN_DIR=build/bin 2024-04-03T16:22:57.0450059Z + SHARD_NUMBER=3 2024-04-03T16:22:57.0450646Z + NUM_TEST_SHARDS=5 2024-04-03T16:22:57.0451387Z + export VALGRIND=ON 2024-04-03T16:22:57.0452186Z + VALGRIND=ON 2024-04-03T16:22:57.0453343Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *clang9* ]] 2024-04-03T16:22:57.0454483Z + [[ 0 == \1 ]] 2024-04-03T16:22:57.0455221Z + [[ False == \1 ]] 2024-04-03T16:22:57.0456306Z + [[ linux-focal-cuda12.1-py3.10-gcc9 != *bazel* ]] 2024-04-03T16:22:57.0457426Z ++ realpath build/custom_test_artifacts 2024-04-03T16:22:57.0460863Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-04-03T16:22:57.0461679Z + [[ -n '' ]] 2024-04-03T16:22:57.0463931Z ++ dirname .ci/pytorch/test.sh 2024-04-03T16:22:57.0471806Z + source .ci/pytorch/common.sh 2024-04-03T16:22:57.0475625Z +++ dirname .ci/pytorch/common.sh 2024-04-03T16:22:57.0484174Z ++ source .ci/pytorch/common_utils.sh 2024-04-03T16:22:57.0486307Z +++ declare -f -t trap_add 2024-04-03T16:22:57.0494311Z ++ set -ex 2024-04-03T16:22:57.0495227Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9 == *rocm* ]] 2024-04-03T16:22:57.0495864Z ++ BUILD_TEST_LIBTORCH=0 2024-04-03T16:22:57.0496223Z + echo 'Environment variables' 2024-04-03T16:22:57.0496700Z Environment variables 2024-04-03T16:22:57.0497074Z + env 2024-04-03T16:22:57.0502482Z INSTALLED_DB=yes 2024-04-03T16:22:57.0503309Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-04-03T16:22:57.0504203Z NVIDIA_VISIBLE_DEVICES=all 2024-04-03T16:22:57.0505147Z NV_NVML_DEV_VERSION=12.1.105-1 2024-04-03T16:22:57.0506661Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-04-03T16:22:57.0508047Z CONTINUE_THROUGH_ERROR=False 2024-04-03T16:22:57.0509169Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-04-03T16:22:57.0510556Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-04-03T16:22:57.0511353Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9 2024-04-03T16:22:57.0512443Z HOSTNAME=5f8b51bc22a6 2024-04-03T16:22:57.0514187Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0516298Z GITHUB_ACTION=__self 2024-04-03T16:22:57.0517132Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-04-03T16:22:57.0524068Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-04-03T16:22:57.0532466Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-04-03T16:22:57.0533120Z NV_NVTX_VERSION=12.1.105-1 2024-04-03T16:22:57.0533540Z GITHUB_RUN_NUMBER=192590 2024-04-03T16:22:57.0533962Z TEST_CONFIG=default 2024-04-03T16:22:57.0534592Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-04-03T16:22:57.0535216Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-04-03T16:22:57.0535790Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-04-03T16:22:57.0536344Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-04-03T16:22:57.0536860Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-04-03T16:22:57.0537334Z GITHUB_TRIGGERING_ACTOR=soulitzer 2024-04-03T16:22:57.0537901Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-04-03T16:22:57.0538467Z GITHUB_REF_TYPE=branch 2024-04-03T16:22:57.0538890Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-04-03T16:22:57.0539363Z NCCL_VERSION=2.17.1-1 2024-04-03T16:22:57.0539821Z BASE_SHA=1d52c2d98507759a2143d52b922c541dbe36b6bf 2024-04-03T16:22:57.0540366Z XLA_CUDA= 2024-04-03T16:22:57.0540877Z HUGGING_FACE_HUB_TOKEN= 2024-04-03T16:22:57.0541372Z *** 2024-04-03T16:22:57.0541694Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-04-03T16:22:57.0542180Z GITHUB_REPOSITORY_ID=65600975 2024-04-03T16:22:57.0542634Z GITHUB_ACTIONS=true 2024-04-03T16:22:57.0543038Z NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:22:57.0543681Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-04-03T16:22:57.0544329Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-04-03T16:22:57.0544899Z SHA1=d253bd7befc64137de913dbb8727b511f1052411 2024-04-03T16:22:57.0545539Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-04-03T16:22:57.0546150Z GITHUB_SHA=4302cb7ecb2b1ae758f931ad18aeedf6ae927cb0 2024-04-03T16:22:57.0547031Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/123154/merge 2024-04-03T16:22:57.0547833Z UCC_HOME=/usr 2024-04-03T16:22:57.0548249Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-04-03T16:22:57.0548718Z VERBOSE_TEST_LOGS=False 2024-04-03T16:22:57.0549140Z NVIDIA_PRODUCT_NAME=CUDA 2024-04-03T16:22:57.0549721Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-04-03T16:22:57.0550288Z GITHUB_REF=refs/pull/123154/merge 2024-04-03T16:22:57.0550807Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-04-03T16:22:57.0551272Z SHARD_NUMBER=3 2024-04-03T16:22:57.0551636Z GITHUB_REF_PROTECTED=false 2024-04-03T16:22:57.0552073Z HOME=/var/lib/jenkins 2024-04-03T16:22:57.0552516Z GITHUB_API_URL=https://api.github.com 2024-04-03T16:22:57.0553038Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-04-03T16:22:57.0553625Z UCX_COMMIT=00bcc6bb18fc282eb160623b4c0d300147f579af 2024-04-03T16:22:57.0554217Z SCCACHE_S3_KEY_PREFIX=pull 2024-04-03T16:22:57.0554695Z CUDA_VERSION=12.1.1 2024-04-03T16:22:57.0555199Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-04-03T16:22:57.0555744Z NUM_TEST_SHARDS=5 2024-04-03T16:22:57.0556091Z UCX_HOME=/usr 2024-04-03T16:22:57.0556733Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-04-03T16:22:57.0558117Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0560118Z JOB_NAME=linux-focal-cuda12.1-py3.10-gcc9 / test (default, 3, 5, linux.4xlarge.nvidia.gpu) 2024-04-03T16:22:57.0561663Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0563097Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-04-03T16:22:57.0563939Z GITHUB_EVENT_NAME=pull_request 2024-04-03T16:22:57.0564371Z DASHBOARD_TAG= 2024-04-03T16:22:57.0564738Z GITHUB_RUN_ID=8536950919 2024-04-03T16:22:57.0565303Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-04-03T16:22:57.0565950Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-04-03T16:22:57.0567221Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0568329Z GITHUB_ACTOR=caic99 2024-04-03T16:22:57.0568753Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-04-03T16:22:57.0569239Z PR_NUMBER=123154 2024-04-03T16:22:57.0569619Z GITHUB_RUN_ATTEMPT=2 2024-04-03T16:22:57.0569994Z VALGRIND=ON 2024-04-03T16:22:57.0570359Z ANACONDA_PYTHON_VERSION=3.10 2024-04-03T16:22:57.0571082Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-04-03T16:22:57.0571666Z TERM=xterm 2024-04-03T16:22:57.0572086Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-04-03T16:22:57.0572600Z INSTALLED_VISION=yes 2024-04-03T16:22:57.0572978Z BRANCH=pull/123154 2024-04-03T16:22:57.0573372Z OPENSSL_ROOT_DIR=/opt/openssl 2024-04-03T16:22:57.0573862Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-04-03T16:22:57.0574372Z CUDA_PATH=/usr/local/cuda 2024-04-03T16:22:57.0575325Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-04-03T16:22:57.0576274Z GITHUB_SERVER_URL=https://github.com 2024-04-03T16:22:57.0576860Z UCC_COMMIT=7cb07a76ccedad7e56ceb136b865eb9319c258ea 2024-04-03T16:22:57.0577446Z REENABLED_ISSUES= 2024-04-03T16:22:57.0577808Z SHLVL=1 2024-04-03T16:22:57.0578287Z MAX_JOBS=14 2024-04-03T16:22:57.0578680Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-04-03T16:22:57.0579117Z NVARCH=x86_64 2024-04-03T16:22:57.0579461Z GITHUB_ACTOR_ID=78061359 2024-04-03T16:22:57.0580042Z GITHUB_WORKFLOW_SHA=4302cb7ecb2b1ae758f931ad18aeedf6ae927cb0 2024-04-03T16:22:57.0580700Z GITHUB_REF_NAME=123154/merge 2024-04-03T16:22:57.0581218Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-04-03T16:22:57.0582044Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-04-03T16:22:57.0582761Z GITHUB_JOB=test 2024-04-03T16:22:57.0583235Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-04-03T16:22:57.0583926Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-04-03T16:22:57.0584566Z NO_TEST_TIMEOUT=False 2024-04-03T16:22:57.0585016Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-04-03T16:22:57.0585557Z GITHUB_REPOSITORY=pytorch/pytorch 2024-04-03T16:22:57.0586080Z NV_NVPROF_VERSION=12.1.105-1 2024-04-03T16:22:57.0586515Z GITHUB_RETENTION_DAYS=90 2024-04-03T16:22:57.0586955Z OPENSSL_DIR=/opt/openssl 2024-04-03T16:22:57.0587385Z GITHUB_ACTION_REPOSITORY= 2024-04-03T16:22:57.0588698Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-04-03T16:22:57.0590074Z GITHUB_BASE_REF=main 2024-04-03T16:22:57.0590491Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-04-03T16:22:57.0590921Z CI=true 2024-04-03T16:22:57.0591310Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-04-03T16:22:57.0591827Z GITHUB_REPOSITORY_OWNER=pytorch 2024-04-03T16:22:57.0592267Z JOB_ID=23402695647 2024-04-03T16:22:57.0592650Z INSTALLED_PROTOBUF=yes 2024-04-03T16:22:57.0593095Z GITHUB_HEAD_REF=patch-1 2024-04-03T16:22:57.0593487Z GITHUB_ACTION_REF= 2024-04-03T16:22:57.0594028Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-04-03T16:22:57.0594616Z GITHUB_WORKFLOW=pull 2024-04-03T16:22:57.0595019Z DEBIAN_FRONTEND=noninteractive 2024-04-03T16:22:57.0596237Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_e4b8e5b7-3e1f-49cc-a6a1-2afce66be39e 2024-04-03T16:22:57.0597298Z NO_TD=False 2024-04-03T16:22:57.0597650Z SKIP_SCCACHE_INITIALIZATION=1 2024-04-03T16:22:57.0598100Z _=/usr/bin/env 2024-04-03T16:22:57.0598531Z + echo 'Testing pytorch' 2024-04-03T16:22:57.0598937Z Testing pytorch 2024-04-03T16:22:57.0599961Z + export LANG=C.UTF-8 2024-04-03T16:22:57.0600393Z + LANG=C.UTF-8 2024-04-03T16:22:57.0600738Z + PR_NUMBER=123154 2024-04-03T16:22:57.0601139Z + [[ default == \d\e\f\a\u\l\t ]] 2024-04-03T16:22:57.0601627Z + export CUDA_VISIBLE_DEVICES=0 2024-04-03T16:22:57.0602082Z + CUDA_VISIBLE_DEVICES=0 2024-04-03T16:22:57.0602516Z + export HIP_VISIBLE_DEVICES=0 2024-04-03T16:22:57.0602976Z + HIP_VISIBLE_DEVICES=0 2024-04-03T16:22:57.0603408Z + [[ default == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-04-03T16:22:57.0603927Z + [[ default == \s\l\o\w ]] 2024-04-03T16:22:57.0604601Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *slow-gradcheck* ]] 2024-04-03T16:22:57.0605380Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *cuda* ]] 2024-04-03T16:22:57.0606045Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-04-03T16:22:57.0606633Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-04-03T16:22:57.0607133Z + [[ default == *crossref* ]] 2024-04-03T16:22:57.0607905Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *rocm* ]] 2024-04-03T16:22:57.0608679Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *xpu* ]] 2024-04-03T16:22:57.0609408Z + [[ linux-focal-cuda12.1-py3.10-gcc9 != *-bazel-* ]] 2024-04-03T16:22:57.0610088Z + pip_install --user ninja==1.10.2 2024-04-03T16:22:57.0610755Z + pip install --progress-bar off --user ninja==1.10.2 2024-04-03T16:22:57.5739346Z Collecting ninja==1.10.2 2024-04-03T16:22:57.5954695Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-04-03T16:22:57.6143021Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-04-03T16:22:58.6541216Z Installing collected packages: ninja 2024-04-03T16:22:58.6640813Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-04-03T16:22:58.6642449Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-04-03T16:22:58.6694769Z Successfully installed ninja-1.10.2 2024-04-03T16:22:58.7426990Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-04-03T16:22:58.7429699Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-04-03T16:22:58.7431271Z + install_tlparse 2024-04-03T16:22:58.7431847Z + pip_install --user tlparse==0.3.7 2024-04-03T16:22:58.7432514Z + pip install --progress-bar off --user tlparse==0.3.7 2024-04-03T16:22:59.2521473Z Collecting tlparse==0.3.7 2024-04-03T16:22:59.2723788Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-04-03T16:22:59.2835871Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-04-03T16:23:00.4014320Z Installing collected packages: tlparse 2024-04-03T16:23:00.4509844Z Successfully installed tlparse-0.3.7 2024-04-03T16:23:00.5240020Z ++ python -m site --user-base 2024-04-03T16:23:00.5467638Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-04-03T16:23:00.5470464Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *asan* ]] 2024-04-03T16:23:00.5471097Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *-debug* ]] 2024-04-03T16:23:00.5471970Z + [[ linux-focal-cuda12.1-py3.10-gcc9 != *-bazel-* ]] 2024-04-03T16:23:00.5473453Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9. Expect the assertion to pass' 2024-04-03T16:23:00.5475335Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9. Expect the assertion to pass 2024-04-03T16:23:00.5476941Z + cd test 2024-04-03T16:23:00.5478315Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-04-03T16:23:02.4340099Z + [[ default == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-04-03T16:23:02.4341006Z + [[ default == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-04-03T16:23:02.4344259Z + DYNAMO_BENCHMARK_FLAGS=() 2024-04-03T16:23:02.4344896Z + [[ default == *dynamo_eager* ]] 2024-04-03T16:23:02.4345395Z + [[ default == *aot_eager* ]] 2024-04-03T16:23:02.4345744Z + [[ default == *aot_inductor* ]] 2024-04-03T16:23:02.4346180Z + [[ default == *inductor* ]] 2024-04-03T16:23:02.4346646Z + [[ default == *dynamic* ]] 2024-04-03T16:23:02.4347103Z + [[ default == *cpu_inductor* ]] 2024-04-03T16:23:02.4347790Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-04-03T16:23:02.4362864Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *tbb* ]] 2024-04-03T16:23:02.4383839Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *libtorch* ]] 2024-04-03T16:23:02.4384509Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *-bazel-* ]] 2024-04-03T16:23:02.4387364Z + cd test 2024-04-03T16:23:02.4388208Z + python -c 'import torch; print(torch.__config__.show())' 2024-04-03T16:23:04.0786197Z PyTorch built with: 2024-04-03T16:23:04.0787315Z - GCC 9.4 2024-04-03T16:23:04.0787796Z - C++ Version: 201703 2024-04-03T16:23:04.0788902Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-04-03T16:23:04.0790280Z - Intel(R) MKL-DNN v3.3.6 (Git Hash 86e6af5974177e513fd3fee58425e1063e7f1361) 2024-04-03T16:23:04.0791115Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-04-03T16:23:04.0791738Z - LAPACK is enabled (usually provided by MKL) 2024-04-03T16:23:04.0792340Z - NNPACK is enabled 2024-04-03T16:23:04.0792814Z - CPU capability usage: AVX2 2024-04-03T16:23:04.0793302Z - CUDA Runtime 12.1 2024-04-03T16:23:04.0794336Z - NVCC architecture flags: -gencode;arch=compute_52,code=sm_52 2024-04-03T16:23:04.0795098Z - CuDNN 8.9.2 2024-04-03T16:23:04.0795411Z - Magma 2.6.1 2024-04-03T16:23:04.0805228Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.9.2, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.4.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-04-03T16:23:04.0813988Z 2024-04-03T16:23:04.3216532Z + cd test 2024-04-03T16:23:04.3217215Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-04-03T16:23:05.9407841Z ATen/Parallel: 2024-04-03T16:23:05.9408737Z at::get_num_threads() : 8 2024-04-03T16:23:05.9409510Z at::get_num_interop_threads() : 16 2024-04-03T16:23:05.9410539Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-04-03T16:23:05.9411509Z omp_get_max_threads() : 8 2024-04-03T16:23:05.9412709Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-04-03T16:23:05.9413730Z mkl_get_max_threads() : 8 2024-04-03T16:23:05.9414501Z Intel(R) MKL-DNN v3.3.6 (Git Hash 86e6af5974177e513fd3fee58425e1063e7f1361) 2024-04-03T16:23:05.9415306Z std::thread::hardware_concurrency() : 16 2024-04-03T16:23:05.9415841Z Environment variables: 2024-04-03T16:23:05.9416263Z OMP_NUM_THREADS : [not set] 2024-04-03T16:23:05.9416719Z MKL_NUM_THREADS : [not set] 2024-04-03T16:23:05.9417178Z ATen parallel backend: OpenMP 2024-04-03T16:23:05.9417496Z 2024-04-03T16:23:06.1878409Z + [[ default == *backward* ]] 2024-04-03T16:23:06.1879669Z + [[ default == *xla* ]] 2024-04-03T16:23:06.1880572Z + [[ default == *executorch* ]] 2024-04-03T16:23:06.1881254Z + [[ default == \j\i\t\_\l\e\g\a\c\y ]] 2024-04-03T16:23:06.1882965Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *libtorch* ]] 2024-04-03T16:23:06.1883815Z + [[ default == distributed ]] 2024-04-03T16:23:06.1884265Z + [[ default == deploy ]] 2024-04-03T16:23:06.1884737Z + [[ default == *inductor_distributed* ]] 2024-04-03T16:23:06.1885515Z + [[ default == *huggingface* ]] 2024-04-03T16:23:06.1885983Z + [[ default == *timm* ]] 2024-04-03T16:23:06.1886423Z + [[ default == *torchbench* ]] 2024-04-03T16:23:06.1887417Z + [[ default == *inductor_cpp_wrapper_abi_compatible* ]] 2024-04-03T16:23:06.1888081Z + [[ default == *inductor* ]] 2024-04-03T16:23:06.1888538Z + [[ default == *dynamo* ]] 2024-04-03T16:23:06.1888978Z + [[ default == *dynamo* ]] 2024-04-03T16:23:06.1889384Z + [[ 3 == 1 ]] 2024-04-03T16:23:06.1889731Z + [[ 3 == 2 ]] 2024-04-03T16:23:06.1890151Z + [[ 3 -gt 2 ]] 2024-04-03T16:23:06.1890507Z + install_torchvision 2024-04-03T16:23:06.1890913Z + local orig_preload 2024-04-03T16:23:06.1891300Z + local commit 2024-04-03T16:23:06.1891658Z ++ get_pinned_commit vision 2024-04-03T16:23:06.1892136Z ++ cat .github/ci_commit_pins/vision.txt 2024-04-03T16:23:06.1899942Z + commit=2c4665ffbb64f03f5d18016d3398af4ac4da5f03 2024-04-03T16:23:06.1900718Z + orig_preload= 2024-04-03T16:23:06.1901404Z + '[' -n '' ']' 2024-04-03T16:23:06.1902482Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@2c4665ffbb64f03f5d18016d3398af4ac4da5f03 2024-04-03T16:23:06.1904365Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@2c4665ffbb64f03f5d18016d3398af4ac4da5f03 2024-04-03T16:23:06.6684698Z Collecting git+https://github.com/pytorch/vision.git@2c4665ffbb64f03f5d18016d3398af4ac4da5f03 2024-04-03T16:23:06.6692627Z Cloning https://github.com/pytorch/vision.git (to revision 2c4665ffbb64f03f5d18016d3398af4ac4da5f03) to /tmp/pip-req-build-esh2crdf 2024-04-03T16:23:06.6715042Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-esh2crdf 2024-04-03T16:23:08.3349625Z Running command git rev-parse -q --verify 'sha^2c4665ffbb64f03f5d18016d3398af4ac4da5f03' 2024-04-03T16:23:08.3373724Z Running command git fetch -q https://github.com/pytorch/vision.git 2c4665ffbb64f03f5d18016d3398af4ac4da5f03 2024-04-03T16:23:09.9304573Z Running command git checkout -q 2c4665ffbb64f03f5d18016d3398af4ac4da5f03 2024-04-03T16:23:10.1822314Z Resolved https://github.com/pytorch/vision.git to commit 2c4665ffbb64f03f5d18016d3398af4ac4da5f03 2024-04-03T16:23:12.7786062Z Preparing metadata (setup.py) ... [?25l- \ done 2024-04-03T16:23:12.7849514Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+2c4665f) (1.21.2) 2024-04-03T16:23:12.7854490Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+2c4665f) (2.4.0a0+gitd253bd7) 2024-04-03T16:23:12.7863229Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+2c4665f) (10.2.0) 2024-04-03T16:23:12.8159996Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+2c4665f) (3.13.1) 2024-04-03T16:23:12.8173877Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+2c4665f) (4.10.0) 2024-04-03T16:23:12.8178688Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+2c4665f) (1.12) 2024-04-03T16:23:12.8183900Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+2c4665f) (2.8.8) 2024-04-03T16:23:12.8189020Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+2c4665f) (3.1.3) 2024-04-03T16:23:12.8194176Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+2c4665f) (2024.2.0) 2024-04-03T16:23:12.8807214Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+2c4665f) (2.1.5) 2024-04-03T16:23:12.9033249Z Requirement already satisfied: mpmath>=0.19 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchvision==0.19.0a0+2c4665f) (1.3.0) 2024-04-03T16:23:12.9142936Z Building wheels for collected packages: torchvision 2024-04-03T16:24:58.4699914Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-04-03T16:24:58.4782751Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+2c4665f-cp310-cp310-linux_x86_64.whl size=2102702 sha256=9c7d43afa312049eae34bbb4777a766d6a11d70ce6819096757c1faddeda8924 2024-04-03T16:24:58.4784904Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/fe/8e/eb/30dab9378a6be22c0f3a178e38ae735f03d8debd545032e10c 2024-04-03T16:24:58.4834670Z Successfully built torchvision 2024-04-03T16:24:59.3786310Z Installing collected packages: torchvision 2024-04-03T16:24:59.9380304Z Successfully installed torchvision-0.19.0a0+2c4665f 2024-04-03T16:25:00.0602639Z + '[' -n '' ']' 2024-04-03T16:25:00.0603163Z + test_python_shard 3 2024-04-03T16:25:00.0603509Z + [[ -z 5 ]] 2024-04-03T16:25:00.0604437Z + python test/run_test.py --exclude-jit-executor --exclude-distributed-tests --shard 3 5 --verbose 2024-04-03T16:25:00.1725267Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-04-03T16:25:00.1726729Z import pkg_resources 2024-04-03T16:25:03.3702003Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-04-03T16:25:03.4415578Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-04-03T16:25:03.4928528Z Ignoring disabled issues: [''] 2024-04-03T16:25:03.5015009Z Found test times from artifacts 2024-04-03T16:25:03.5348983Z Found test times from artifacts 2024-04-03T16:25:03.5358506Z Running all tests 2024-04-03T16:25:03.5747974Z Name: tests to run (est. time: 90.67min) 2024-04-03T16:25:03.5748526Z Serial tests (0): 2024-04-03T16:25:03.5748918Z Parallel tests (142): 2024-04-03T16:25:03.5749338Z test_foreach 1/2 2024-04-03T16:25:03.5754674Z test_ops_jit 2/3 2024-04-03T16:25:03.5755536Z export/test_functionalized_assertions 1/1 2024-04-03T16:25:03.5756112Z dynamo/test_fx_passes_pre_grad 1/1 2024-04-03T16:25:03.5756644Z test_comparison_utils 1/1 2024-04-03T16:25:03.5757116Z inductor/test_smoke 1/1 2024-04-03T16:25:03.5757835Z inductor/test_config 1/1 2024-04-03T16:25:03.5758591Z test_mkl_verbose 1/1 2024-04-03T16:25:03.5759887Z test_package 1/1 2024-04-03T16:25:03.5760831Z inductor/test_extension_backend 1/1 2024-04-03T16:25:03.5761677Z test_mkldnn_verbose 1/1 2024-04-03T16:25:03.5762166Z export/test_tree_utils 1/1 2024-04-03T16:25:03.5762651Z inductor/test_indexing 1/1 2024-04-03T16:25:03.5763106Z test_ao_sparsity 1/1 2024-04-03T16:25:03.5763540Z inductor/test_fx_fusion 1/1 2024-04-03T16:25:03.5763991Z inductor/test_utils 1/1 2024-04-03T16:25:03.5764478Z inductor/test_metrics 1/1 2024-04-03T16:25:03.5764924Z test_license 1/1 2024-04-03T16:25:03.5765360Z inductor/test_triton_extension_backend 1/1 2024-04-03T16:25:03.5766124Z dynamo/test_sdpa 1/1 2024-04-03T16:25:03.5766958Z inductor/test_torchinductor_opinfo 1/1 2024-04-03T16:25:03.5768027Z dynamo/test_base_output 1/1 2024-04-03T16:25:03.5769055Z inductor/test_dependencies 1/1 2024-04-03T16:25:03.5770109Z dynamo/test_bytecode_hook 1/1 2024-04-03T16:25:03.5770600Z dynamo/test_frame_init 1/1 2024-04-03T16:25:03.5771098Z dynamo/test_cpp_guard_manager 1/1 2024-04-03T16:25:03.5771645Z inductor/test_aot_inductor_utils 1/1 2024-04-03T16:25:03.5772500Z inductor/test_coordinate_descent_tuner 1/1 2024-04-03T16:25:03.5773048Z dynamo/test_compile 1/1 2024-04-03T16:25:03.5773874Z dynamo/test_guard_manager 1/1 2024-04-03T16:25:03.5774722Z lazy/test_functionalization 1/1 2024-04-03T16:25:03.5775607Z test_tensorexpr_pybind 1/1 2024-04-03T16:25:03.5776402Z inductor/test_debug_trace 1/1 2024-04-03T16:25:03.5776869Z inductor/test_inductor_freezing 1/1 2024-04-03T16:25:03.5777697Z inductor/test_profiler 1/1 2024-04-03T16:25:03.5778225Z test_native_functions 1/1 2024-04-03T16:25:03.5778994Z inductor/test_triton_kernels 1/1 2024-04-03T16:25:03.5779517Z inductor/test_minifier_isolate 1/1 2024-04-03T16:25:03.5780305Z dynamo/test_sources 1/1 2024-04-03T16:25:03.5780774Z inductor/test_codecache 1/1 2024-04-03T16:25:03.5781379Z torch_np/test_dtype 1/1 2024-04-03T16:25:03.5781961Z export/test_db 1/1 2024-04-03T16:25:03.5782391Z dynamo/test_decorators 1/1 2024-04-03T16:25:03.5783133Z inductor/test_control_flow 1/1 2024-04-03T16:25:03.5783857Z dynamo/test_optimizers 1/1 2024-04-03T16:25:03.5784645Z test_functionalization_of_rng_ops 1/1 2024-04-03T16:25:03.5785150Z test_ops 1/9 2024-04-03T16:25:03.5785789Z test_ops 6/9 2024-04-03T16:25:03.5786181Z test_ops 7/9 2024-04-03T16:25:03.5786548Z test_ops_gradients 3/3 2024-04-03T16:25:03.5787333Z test_set_default_mobile_cpu_allocator 1/1 2024-04-03T16:25:03.5787918Z inductor/test_compiled_autograd 1/1 2024-04-03T16:25:03.5788710Z inductor/test_perf 1/1 2024-04-03T16:25:03.5789161Z export/test_schema 1/1 2024-04-03T16:25:03.5789838Z test_proxy_tensor 1/1 2024-04-03T16:25:03.5790303Z export/test_experimental 1/1 2024-04-03T16:25:03.5790990Z torch_np/numpy_tests/lib/test_histograms 1/1 2024-04-03T16:25:03.5791706Z export/test_torchbind 1/1 2024-04-03T16:25:03.5792237Z torch_np/test_basic 1/1 2024-04-03T16:25:03.5792868Z test_optim 1/1 2024-04-03T16:25:03.5793274Z export/test_lift_unlift 1/1 2024-04-03T16:25:03.5794033Z backends/xeon/test_launch 1/1 2024-04-03T16:25:03.5794547Z lazy/test_step_closures 1/1 2024-04-03T16:25:03.5795213Z inductor/test_mmdecomp 1/1 2024-04-03T16:25:03.5795751Z test_typing 1/1 2024-04-03T16:25:03.5796190Z inductor/test_pattern_matcher 1/1 2024-04-03T16:25:03.5796987Z test_per_overload_api 1/1 2024-04-03T16:25:03.5797432Z test_stateless 1/1 2024-04-03T16:25:03.5798129Z test_public_bindings 1/1 2024-04-03T16:25:03.5798590Z test_pytree 1/1 2024-04-03T16:25:03.5799029Z torch_np/test_nep50_examples 1/1 2024-04-03T16:25:03.5799964Z nn/test_packed_sequence 1/1 2024-04-03T16:25:03.5800592Z inductor/test_group_batch_fusion 1/1 2024-04-03T16:25:03.5801290Z export/test_export_nonstrict 1/1 2024-04-03T16:25:03.5801900Z test_expanded_weights 1/1 2024-04-03T16:25:03.5802577Z dynamo/test_input_attr_tracking 1/1 2024-04-03T16:25:03.5803151Z test_functional_autograd_benchmark 1/1 2024-04-03T16:25:03.5803951Z test_flop_counter 1/1 2024-04-03T16:25:03.5804405Z dynamo/test_export 1/1 2024-04-03T16:25:03.5805131Z torch_np/test_random 1/1 2024-04-03T16:25:03.5805591Z test_sympy_utils 1/1 2024-04-03T16:25:03.5806210Z export/test_upgrade 1/1 2024-04-03T16:25:03.5806759Z dynamo/test_profiler 1/1 2024-04-03T16:25:03.5807277Z dynamo/test_verify_correctness 1/1 2024-04-03T16:25:03.5808049Z test_out_dtype_op 1/1 2024-04-03T16:25:03.5808492Z inductor/test_memory_planning 1/1 2024-04-03T16:25:03.5809330Z higher_order_ops/test_with_effects 1/1 2024-04-03T16:25:03.5809884Z dynamo/test_ctx_manager 1/1 2024-04-03T16:25:03.5810629Z export/test_pass_infra 1/1 2024-04-03T16:25:03.5811160Z inductor/test_custom_post_grad_passes 1/1 2024-04-03T16:25:03.5812005Z dynamo/test_debug_utils 1/1 2024-04-03T16:25:03.5812463Z test_matmul_cuda 1/1 2024-04-03T16:25:03.5813188Z dynamo/test_backward_higher_order_ops 1/1 2024-04-03T16:25:03.5813787Z test_numba_integration 1/1 2024-04-03T16:25:03.5814450Z test_fx_experimental 1/1 2024-04-03T16:25:03.5815027Z dynamo/test_higher_order_ops 1/1 2024-04-03T16:25:03.5815674Z functorch/test_vmap_registrations 1/1 2024-04-03T16:25:03.5816433Z torch_np/numpy_tests/core/test_getlimits 1/1 2024-04-03T16:25:03.5817406Z torch_np/test_ufuncs_basic 1/1 2024-04-03T16:25:03.5817952Z test_pruning_op 1/1 2024-04-03T16:25:03.5818479Z test_decomp 1/16 2024-04-03T16:25:03.5819049Z test_decomp 6/16 2024-04-03T16:25:03.5819449Z test_decomp 7/16 2024-04-03T16:25:03.5819826Z test_decomp 12/16 2024-04-03T16:25:03.5820507Z test_decomp 13/16 2024-04-03T16:25:03.5820940Z dynamo/test_torchrec 1/1 2024-04-03T16:25:03.5821385Z inductor/test_cpu_repro 2/2 2024-04-03T16:25:03.5821865Z dynamo/test_trace_rules 1/1 2024-04-03T16:25:03.5822338Z functorch/test_ops 1/8 2024-04-03T16:25:03.5823055Z functorch/test_ops 5/8 2024-04-03T16:25:03.5823506Z functorch/test_ops 8/8 2024-04-03T16:25:03.5823947Z test_maskedtensor 1/1 2024-04-03T16:25:03.5824601Z torch_np/numpy_tests/core/test_scalarmath 1/1 2024-04-03T16:25:03.5825248Z torch_np/numpy_tests/lib/test_function_base 1/1 2024-04-03T16:25:03.5825895Z torch_np/numpy_tests/lib/test_shape_base_ 1/1 2024-04-03T16:25:03.5826445Z test_futures 1/1 2024-04-03T16:25:03.5827230Z torch_np/numpy_tests/core/test_multiarray 1/1 2024-04-03T16:25:03.5827804Z test_testing 1/1 2024-04-03T16:25:03.5828190Z lazy/test_ts_opinfo 1/1 2024-04-03T16:25:03.5828691Z torch_np/numpy_tests/core/test_dlpack 1/1 2024-04-03T16:25:03.5829259Z torch_np/test_ndarray_methods 1/1 2024-04-03T16:25:03.5829752Z functorch/test_dims 1/1 2024-04-03T16:25:03.5830203Z lazy/test_debug_util 1/1 2024-04-03T16:25:03.5830645Z test_mkldnn 1/1 2024-04-03T16:25:03.5831092Z torch_np/numpy_tests/lib/test_index_tricks 1/1 2024-04-03T16:25:03.5831736Z torch_np/numpy_tests/lib/test_twodim_base 1/1 2024-04-03T16:25:03.5832298Z test_datapipe 1/1 2024-04-03T16:25:03.5832683Z test_shape_ops 1/1 2024-04-03T16:25:03.5833111Z test_nestedtensor 1/1 2024-04-03T16:25:03.5833541Z test_jit_fuser_te 1/2 2024-04-03T16:25:03.5834127Z profiler/test_profiler_tree 1/1 2024-04-03T16:25:03.5834796Z dynamo/test_misc 1/1 2024-04-03T16:25:03.5835253Z torch_np/test_function_base 1/1 2024-04-03T16:25:03.5835735Z test_dynamic_shapes 1/1 2024-04-03T16:25:03.5836244Z torch_np/numpy_tests/lib/test_arraypad 1/1 2024-04-03T16:25:03.5836790Z test_view_ops 1/1 2024-04-03T16:25:03.5837201Z torch_np/test_scalars_0D_arrays 1/1 2024-04-03T16:25:03.5837726Z dynamo/test_comptime 1/1 2024-04-03T16:25:03.5838178Z lazy/test_reuse_ir 1/1 2024-04-03T16:25:03.5838586Z test_sparse 1/2 2024-04-03T16:25:03.5838977Z test_binary_ufuncs 1/1 2024-04-03T16:25:03.5839632Z lazy/test_bindings 1/1 2024-04-03T16:25:03.5840068Z Name: excluded (est. time: 0.0min) 2024-04-03T16:25:03.5840556Z Serial tests (0): 2024-04-03T16:25:03.5840951Z Parallel tests (0): 2024-04-03T16:25:03.5841755Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-04-03T16:25:05.6126935Z Running test_foreach 1/2 ... [2024-04-03 16:25:05.612130] 2024-04-03T16:25:05.6131746Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_foreach.py', '--shard-id=1', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-04-03 16:25:05.612665] 2024-04-03T16:25:05.6165308Z Running test_ops_jit 2/3 ... [2024-04-03 16:25:05.616119] 2024-04-03T16:25:05.6171724Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_ops_jit.py', '--shard-id=2', '--num-shards=3', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-04-03 16:25:05.616664] 2024-04-03T16:26:27.5311256Z 2024-04-03T16:26:27.5313051Z PRINTING LOG FILE of test_foreach 1/2 (test/test-reports/test_foreach_1.2_67e79d7959fa5e16_.log) 2024-04-03T16:26:27.5316249Z Test results will be stored in test-reports/python-pytest/test_foreach/test_foreach-9cbc8d00a0d5d7a6.xml 2024-04-03T16:26:27.5320660Z ============================= test session starts ============================== 2024-04-03T16:26:27.5323403Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.4.0 -- /opt/conda/envs/py_3.10/bin/python 2024-04-03T16:26:27.5324269Z cachedir: .pytest_cache 2024-04-03T16:26:27.5325814Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-04-03T16:26:27.5327200Z rootdir: /var/lib/jenkins/workspace 2024-04-03T16:26:27.5328126Z configfile: pytest.ini 2024-04-03T16:26:27.5329780Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-04-03T16:26:27.5331366Z collecting ... collected 2190 items 2024-04-03T16:26:27.5331887Z stepcurrent: Cannot find last run test, not skipping 2024-04-03T16:26:27.6134161Z Running 1116 items in this shard: test/test_foreach.py::TestForeachCUDA::test_0dim_tensor_overload_cpu_ok_cuda, test/test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_abs_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_acos_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_add_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_asin_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_ceil_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_clamp_max_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_clamp_min_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_cos_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_expm1_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_frac_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_lerp_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_log10_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_log2_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_maximum_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_mul_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_neg_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_norm_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_pow_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_reciprocal_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_sub_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_abs_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_abs_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_abs_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_acos_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_add_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_addcdiv_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_addcdiv_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_addcmul_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_addcmul_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_asin_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_asin_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_asin_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_atan_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_atan_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_atan_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_atan_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_ceil_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_clamp_min_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_copy_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_copy_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cos_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cos_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cos_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cosh_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cosh_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_div_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_div_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_erfc_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_erfc_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_exp_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_exp_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_expm1_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_expm1_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_floor_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_frac_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_lerp_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_lerp_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log10_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log1p_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log1p_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log2_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log2_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_maximum_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_minimum_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_mul_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_mul_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_neg_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_neg_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_neg_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_norm_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_pow_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_reciprocal_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_round_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sigmoid_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sign_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sign_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sin_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sin_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sinh_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sqrt_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sub_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_tanh_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_trunc_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_trunc_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_zero_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_zero_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_autodiff__foreach_zero_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_big_num_tensors__foreach_norm_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_big_num_tensors__foreach_norm_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_add_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_clamp_max_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_clamp_min_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_clamp_min_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_clamp_min_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_div_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_div_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_maximum_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_minimum_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_mul_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_mul_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_pow_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_sub_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_sub_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_sub_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_max_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_max_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_max_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_max_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_max_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_min_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_min_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_min_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_min_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_min_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_clamp_min_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_div_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_div_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_div_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_div_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_div_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_div_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_div_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_div_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_div_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_maximum_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_maximum_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_maximum_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_maximum_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_maximum_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_maximum_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_maximum_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_minimum_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_minimum_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_minimum_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_minimum_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_minimum_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_minimum_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_minimum_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_mul_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_mul_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_mul_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_mul_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_mul_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_mul_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_pow_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_pow_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_pow_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_pow_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_sub_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_sub_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_sub_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_sub_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_sub_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_sub_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_add_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_add_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_add_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_add_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_add_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_add_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_max_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_max_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_max_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_max_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_min_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_min_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_min_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_min_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_min_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_clamp_min_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_div_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_div_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_div_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_div_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_div_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_div_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_maximum_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_maximum_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_maximum_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_maximum_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_maximum_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_minimum_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_minimum_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_minimum_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_minimum_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_mul_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_pow_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_pow_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_pow_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_pow_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_pow_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_sub_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_sub_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_sub_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_sub_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_sub_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_list_slow_path__foreach_sub_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_different_tensor_dtypes__foreach_clamp_max_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_different_tensor_dtypes__foreach_div_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_add_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_add_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_add_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_add_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_add_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_add_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_add_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_add_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_clamp_max_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_clamp_max_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_clamp_max_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_clamp_max_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_clamp_min_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_clamp_min_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_clamp_min_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_clamp_min_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_div_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_div_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_div_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_div_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_div_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_maximum_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_maximum_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_minimum_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_minimum_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_minimum_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_minimum_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_minimum_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_mul_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_mul_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_mul_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_pow_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_pow_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_pow_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_sub_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_sub_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_sub_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_scalar_with_overlapping_tensors__foreach_sub_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_add_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_add_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_add_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_add_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_add_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_add_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_add_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_clamp_max_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_clamp_min_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_clamp_min_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_clamp_min_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_clamp_min_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_clamp_min_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_clamp_min_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_div_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_div_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_div_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_maximum_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_maximum_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_maximum_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_maximum_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_maximum_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_maximum_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_minimum_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_minimum_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_minimum_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_minimum_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_minimum_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_minimum_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_mul_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_mul_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_mul_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_mul_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_mul_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_mul_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_mul_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_pow_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_pow_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_pow_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_pow_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_sub_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_sub_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_sub_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_sub_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_tensors_on_different_devices__foreach_sub_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_binary_op_with_scalar_self_support__foreach_pow_is_fastpath_False_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_binary_op_with_scalar_self_support__foreach_pow_is_fastpath_False_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_with_scalar_self_support__foreach_pow_is_fastpath_False_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_with_scalar_self_support__foreach_pow_is_fastpath_False_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_binary_op_with_scalar_self_support__foreach_pow_is_fastpath_False_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_binary_op_with_scalar_self_support__foreach_pow_is_fastpath_True_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_binary_op_with_scalar_self_support__foreach_pow_is_fastpath_True_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_binary_op_with_scalar_self_support__foreach_pow_is_fastpath_True_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_binary_op_with_scalar_self_support__foreach_pow_is_fastpath_True_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_foreach_copy_with_multi_device_inputs__foreach_copy_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_foreach_copy_with_multi_device_inputs__foreach_copy_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_foreach_copy_with_multi_device_inputs__foreach_copy_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_foreach_copy_with_multi_device_inputs__foreach_copy_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_foreach_l2_large_value_input__foreach_norm_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_foreach_l2_large_value_input__foreach_norm_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_foreach_reduce_large_input__foreach_norm_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_foreach_reduce_large_input__foreach_norm_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_foreach_reduce_large_input__foreach_norm_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_abs_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_add_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_addcdiv_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_addcmul_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_clamp_max_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_erfc_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_expm1_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_lerp_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_lgamma_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_log1p_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_mul_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_neg_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_pow_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_sigmoid_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_sin_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_sqrt_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_tan_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_trunc_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_inplace_foreach_leaf_check_and_grad_fn__foreach_zero_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_lifetime_of_grad_fn_when_result_is_saved__foreach_exp_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_lifetime_of_grad_fn_when_result_is_saved__foreach_pow_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_lifetime_of_grad_fn_when_result_is_saved__foreach_sigmoid_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_lifetime_of_grad_fn_when_result_is_saved__foreach_sqrt_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_lifetime_of_grad_fn_when_result_is_saved__foreach_tan_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_abs_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_acos_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_add_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_addcdiv_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_asin_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_clamp_max_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_clamp_min_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_cosh_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_erfc_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_floor_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_lgamma_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_log_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_maximum_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_minimum_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_mul_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_neg_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_reciprocal_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_round_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_sign_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_sin_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_sqrt_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_tan_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_outplace_with_invalid_grads__foreach_trunc_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_inplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_outplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_inplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_abs_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_acos_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_inplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_outplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_inplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_outplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_add_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_fastpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcdiv_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_fastpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_addcmul_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_fastpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_asin_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_fastpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_atan_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_fastpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_slowpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_ceil_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_fastpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_fastpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_slowpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_max_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_fastpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_clamp_min_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_copy_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_copy_fastpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_copy_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_copy_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_copy_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_copy_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_copy_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cos_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cos_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cos_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cos_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cos_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cos_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cos_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cos_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cos_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_cosh_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_inplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_outplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_fastpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_outplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_div_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erf_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erf_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erf_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erf_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erf_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erf_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erfc_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erfc_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erfc_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erfc_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erfc_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erfc_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erfc_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_erfc_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_exp_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_fastpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_expm1_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_fastpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_slowpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_floor_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_frac_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lerp_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_inplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_outplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_inplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_outplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_lgamma_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log10_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log10_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log10_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log10_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log10_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log10_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log10_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log10_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log10_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log1p_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log2_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_log_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_fastpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_slowpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_slowpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_maximum_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_slowpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_minimum_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_outplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_fastpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_mul_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_fastpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_fastpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_slowpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_slowpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_neg_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_fastpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_norm_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_slowpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_slowpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_pow_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_reciprocal_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_reciprocal_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_reciprocal_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_reciprocal_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_reciprocal_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_fastpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_slowpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_slowpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_slowpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_slowpath_outplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_slowpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_round_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sigmoid_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sigmoid_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sigmoid_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sigmoid_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sigmoid_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sign_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sign_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sign_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sign_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sign_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sign_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sign_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sign_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sin_fastpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sin_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sin_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sin_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sin_slowpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sin_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sin_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sin_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sinh_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sqrt_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_fastpath_inplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_fastpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_fastpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_outplace_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_sub_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tan_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tan_fastpath_inplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tan_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tan_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tan_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tan_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tan_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tan_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tan_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_fastpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_slowpath_inplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_tanh_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_inplace_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_inplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_outplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_outplace_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_fastpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_inplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_inplace_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_inplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_inplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_outplace_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_outplace_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_trunc_slowpath_outplace_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_fastpath_inplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_fastpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_fastpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_fastpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_slowpath_inplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_slowpath_inplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_slowpath_outplace_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_slowpath_outplace_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_slowpath_outplace_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_slowpath_outplace_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_parity__foreach_zero_slowpath_outplace_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_tensors_on_different_devices__foreach_addcdiv_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_False_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_False_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_False_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_False_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_False_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_False_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_False_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_True_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_True_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_True_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_True_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcdiv_is_fastpath_True_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcmul_is_fastpath_False_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcmul_is_fastpath_False_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcmul_is_fastpath_False_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcmul_is_fastpath_False_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcmul_is_fastpath_False_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcmul_is_fastpath_True_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcmul_is_fastpath_True_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcmul_is_fastpath_True_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_pointwise_op_with_tensor_of_scalarlist_overload__foreach_addcmul_is_fastpath_True_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_tensors_grouping_cuda, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_abs_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_abs_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_abs_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_abs_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_abs_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_abs_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_acos_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_acos_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_acos_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_asin_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_asin_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_asin_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_asin_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_atan_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_atan_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_atan_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_ceil_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_ceil_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_ceil_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_cos_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_cosh_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_cosh_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_erf_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_erf_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_erfc_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_erfc_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_exp_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_exp_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_exp_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_exp_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_exp_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_expm1_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_expm1_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_expm1_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_floor_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_floor_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_floor_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_floor_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_floor_cuda_int32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_floor_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_floor_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_lgamma_cuda_bool, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_lgamma_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_lgamma_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_lgamma_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_lgamma_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log10_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log10_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log10_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log1p_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log1p_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log1p_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log1p_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log2_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log2_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log2_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_log_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_neg_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_neg_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_neg_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_reciprocal_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_reciprocal_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_round_cuda_bfloat16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_round_cuda_int64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_round_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_round_cuda_uint8, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sigmoid_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sigmoid_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sign_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sin_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sin_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sinh_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sinh_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sinh_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sqrt_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_sqrt_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_tan_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_tan_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_tan_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_tanh_cuda_complex128, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_tanh_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_tanh_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_trunc_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_trunc_cuda_float32, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_trunc_cuda_float64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_trunc_cuda_int16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_trunc_cuda_int8, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_zero_cuda_complex64, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_zero_cuda_float16, test/test_foreach.py::TestForeachCUDA::test_unary_op_tensors_on_different_devices__foreach_zero_cuda_float32 2024-04-03T16:26:27.6791759Z 2024-04-03T16:26:27.6792373Z test_foreach.py::TestForeachCUDA::test_0dim_tensor_overload_cpu_ok_cuda PASSED [1.0307s] [ 0%] 2024-04-03T16:26:27.6793778Z test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_bfloat16 PASSED [0.0030s] [ 0%] 2024-04-03T16:26:27.6795259Z test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_float16 PASSED [0.0023s] [ 0%] 2024-04-03T16:26:27.6796750Z test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_float32 PASSED [0.0023s] [ 0%] 2024-04-03T16:26:27.6798218Z test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_int32 PASSED [0.0020s] [ 0%] 2024-04-03T16:26:27.6803647Z test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_int64 PASSED [0.0020s] [ 0%] 2024-04-03T16:26:27.6805087Z test_foreach.py::TestForeachCUDA::test_add_scalar_with_empty_list_and_empty_tensor_cuda_int8 PASSED [0.0020s] [ 0%] 2024-04-03T16:26:27.6806616Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_abs_cuda_float32 PASSED [0.0023s] [ 0%] 2024-04-03T16:26:27.6808258Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_acos_cuda_float32 PASSED [0.0023s] [ 0%] 2024-04-03T16:26:27.6810046Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_add_cuda_float32 PASSED [0.0024s] [ 0%] 2024-04-03T16:26:27.6811745Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_asin_cuda_float32 PASSED [0.0022s] [ 0%] 2024-04-03T16:26:27.6813357Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_ceil_cuda_float32 PASSED [0.0022s] [ 1%] 2024-04-03T16:26:27.6814993Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_clamp_max_cuda_float32 PASSED [0.0022s] [ 1%] 2024-04-03T16:26:27.6816639Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_clamp_min_cuda_float32 PASSED [0.0024s] [ 1%] 2024-04-03T16:26:27.6818431Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_cos_cuda_float32 PASSED [0.0023s] [ 1%] 2024-04-03T16:26:27.6820105Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_expm1_cuda_float32 PASSED [0.0023s] [ 1%] 2024-04-03T16:26:27.6821737Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_frac_cuda_float32 PASSED [0.0023s] [ 1%] 2024-04-03T16:26:27.6823323Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_lerp_cuda_float32 PASSED [0.0022s] [ 1%] 2024-04-03T16:26:27.6824938Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_log10_cuda_float32 PASSED [0.0022s] [ 1%] 2024-04-03T16:26:27.6826556Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_log2_cuda_float32 PASSED [0.0022s] [ 1%] 2024-04-03T16:26:27.6828194Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_maximum_cuda_float32 PASSED [0.0022s] [ 1%] 2024-04-03T16:26:27.6829800Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_mul_cuda_float32 PASSED [0.0023s] [ 1%] 2024-04-03T16:26:27.6831398Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_neg_cuda_float32 PASSED [0.0025s] [ 2%] 2024-04-03T16:26:27.6832996Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_norm_cuda_float32 PASSED [0.0023s] [ 2%] 2024-04-03T16:26:27.6834601Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_pow_cuda_float32 PASSED [0.0023s] [ 2%] 2024-04-03T16:26:27.6836224Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_reciprocal_cuda_float32 PASSED [0.0022s] [ 2%] 2024-04-03T16:26:27.6837861Z test_foreach.py::TestForeachCUDA::test_all_zero_size_tensors_do_not_launch_kernel__foreach_sub_cuda_float32 PASSED [0.0023s] [ 2%] 2024-04-03T16:26:27.6839545Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_abs_inplace_cuda_complex128 PASSED [0.0186s] [ 2%] 2024-04-03T16:26:27.6840901Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_abs_inplace_cuda_float64 PASSED [0.3934s] [ 2%] 2024-04-03T16:26:27.6842240Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_abs_outplace_cuda_float64 PASSED [0.0929s] [ 2%] 2024-04-03T16:26:27.6843610Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_acos_outplace_cuda_complex128 PASSED [1.7899s] [ 2%] 2024-04-03T16:26:27.6844990Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_add_inplace_cuda_complex128 PASSED [5.2680s] [ 2%] 2024-04-03T16:26:27.6846362Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_addcdiv_inplace_cuda_complex128 PASSED [4.3315s] [ 2%] 2024-04-03T16:26:27.6847778Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_addcdiv_outplace_cuda_complex128 PASSED [4.1854s] [ 3%] 2024-04-03T16:26:27.6849208Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_addcmul_outplace_cuda_complex128 PASSED [3.7375s] [ 3%] 2024-04-03T16:26:27.6850726Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_addcmul_outplace_cuda_float64 PASSED [1.2233s] [ 3%] 2024-04-03T16:26:27.6852118Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_asin_inplace_cuda_complex128 PASSED [0.4648s] [ 3%] 2024-04-03T16:26:27.6853476Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_asin_inplace_cuda_float64 PASSED [0.1447s] [ 3%] 2024-04-03T16:26:27.6854822Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_asin_outplace_cuda_float64 PASSED [0.0997s] [ 3%] 2024-04-03T16:26:27.6856163Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_atan_inplace_cuda_complex128 PASSED [0.4484s] [ 3%] 2024-04-03T16:26:27.6857523Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_atan_inplace_cuda_float64 PASSED [0.1090s] [ 3%] 2024-04-03T16:26:27.6859007Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_atan_outplace_cuda_complex128 PASSED [0.4272s] [ 3%] 2024-04-03T16:26:27.6860380Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_atan_outplace_cuda_float64 PASSED [0.0945s] [ 3%] 2024-04-03T16:26:27.6861713Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_ceil_inplace_cuda_float64 PASSED [0.0994s] [ 3%] 2024-04-03T16:26:27.6863082Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_clamp_min_inplace_cuda_float64 PASSED [0.8326s] [ 4%] 2024-04-03T16:26:27.6864790Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_copy_inplace_cuda_complex128 SKIPPED [0.0025s] (neither reverse mode nor forward mode supported) [ 4%] 2024-04-03T16:26:27.6866760Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_copy_inplace_cuda_float64 SKIPPED [0.0023s] (neither reverse mode nor forward mode supported) [ 4%] 2024-04-03T16:26:27.6868396Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cos_inplace_cuda_complex128 PASSED [0.9455s] [ 4%] 2024-04-03T16:26:27.6869751Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cos_inplace_cuda_float64 PASSED [0.1101s] [ 4%] 2024-04-03T16:26:27.6871108Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cos_outplace_cuda_complex128 PASSED [0.4302s] [ 4%] 2024-04-03T16:26:27.6872446Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cosh_inplace_cuda_float64 PASSED [0.1089s] [ 4%] 2024-04-03T16:26:27.6873789Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_cosh_outplace_cuda_float64 PASSED [0.0945s] [ 4%] 2024-04-03T16:26:27.6875148Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_div_outplace_cuda_complex128 PASSED [5.0705s] [ 4%] 2024-04-03T16:26:27.6876498Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_div_outplace_cuda_float64 PASSED [0.8651s] [ 4%] 2024-04-03T16:26:27.6877811Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_erfc_inplace_cuda_float64 PASSED [0.1586s] [ 4%] 2024-04-03T16:26:27.6879318Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_erfc_outplace_cuda_float64 PASSED [0.1025s] [ 5%] 2024-04-03T16:26:27.6880707Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_exp_inplace_cuda_complex128 PASSED [0.4361s] [ 5%] 2024-04-03T16:26:27.6882063Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_exp_inplace_cuda_float64 PASSED [0.1018s] [ 5%] 2024-04-03T16:26:27.6883411Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_expm1_inplace_cuda_complex128 PASSED [0.4401s] [ 5%] 2024-04-03T16:26:27.6884800Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_expm1_outplace_cuda_complex128 PASSED [0.4290s] [ 5%] 2024-04-03T16:26:27.6886172Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_floor_outplace_cuda_float64 PASSED [0.0936s] [ 5%] 2024-04-03T16:26:27.6887502Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_frac_inplace_cuda_float64 PASSED [0.1055s] [ 5%] 2024-04-03T16:26:27.6888851Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_lerp_inplace_cuda_complex128 PASSED [3.6232s] [ 5%] 2024-04-03T16:26:27.6890221Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_lerp_inplace_cuda_float64 PASSED [0.2899s] [ 5%] 2024-04-03T16:26:27.6891744Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log10_inplace_cuda_complex128 PASSED [0.4544s] [ 5%] 2024-04-03T16:26:27.6893136Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log1p_inplace_cuda_complex128 PASSED [0.4592s] [ 5%] 2024-04-03T16:26:27.6894527Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log1p_outplace_cuda_complex128 PASSED [0.4289s] [ 6%] 2024-04-03T16:26:27.6895917Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log2_inplace_cuda_complex128 PASSED [0.4279s] [ 6%] 2024-04-03T16:26:27.6897257Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log2_outplace_cuda_float64 PASSED [0.0884s] [ 6%] 2024-04-03T16:26:27.6898616Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log_inplace_cuda_complex128 PASSED [0.4422s] [ 6%] 2024-04-03T16:26:27.6900104Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log_inplace_cuda_float64 PASSED [0.1027s] [ 6%] 2024-04-03T16:26:27.6901436Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_log_outplace_cuda_float64 PASSED [0.0899s] [ 6%] 2024-04-03T16:26:27.6902781Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_maximum_outplace_cuda_float64 PASSED [0.7611s] [ 6%] 2024-04-03T16:26:27.6904159Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_minimum_outplace_cuda_float64 PASSED [0.4258s] [ 6%] 2024-04-03T16:26:27.6905535Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_mul_inplace_cuda_complex128 PASSED [5.1779s] [ 6%] 2024-04-03T16:26:27.6906880Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_mul_outplace_cuda_float64 PASSED [0.8363s] [ 6%] 2024-04-03T16:26:27.6908211Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_neg_inplace_cuda_complex128 PASSED [0.8602s] [ 6%] 2024-04-03T16:26:27.6909550Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_neg_inplace_cuda_float64 PASSED [0.0995s] [ 6%] 2024-04-03T16:26:27.6910890Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_neg_outplace_cuda_float64 PASSED [0.0902s] [ 7%] 2024-04-03T16:26:27.6912733Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_norm_inplace_cuda_complex128 SKIPPED [0.0023s] (in-place not implemented) [ 7%] 2024-04-03T16:26:27.6914259Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_pow_inplace_cuda_float64 PASSED [0.9430s] [ 7%] 2024-04-03T16:26:27.6915621Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_reciprocal_inplace_cuda_float64 PASSED [0.0993s] [ 7%] 2024-04-03T16:26:27.6917002Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_round_outplace_cuda_float64 PASSED [0.0893s] [ 7%] 2024-04-03T16:26:27.6918347Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sigmoid_inplace_cuda_float64 PASSED [0.1040s] [ 7%] 2024-04-03T16:26:27.6919878Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sign_inplace_cuda_float64 PASSED [0.0964s] [ 7%] 2024-04-03T16:26:27.6921242Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sign_outplace_cuda_float64 PASSED [0.0877s] [ 7%] 2024-04-03T16:26:27.6922562Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sin_inplace_cuda_float64 PASSED [0.1031s] [ 7%] 2024-04-03T16:26:27.6923914Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sin_outplace_cuda_complex128 PASSED [0.8779s] [ 7%] 2024-04-03T16:26:27.6925292Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sinh_outplace_cuda_complex128 PASSED [0.8795s] [ 7%] 2024-04-03T16:26:27.6926665Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sqrt_inplace_cuda_complex128 PASSED [0.4127s] [ 8%] 2024-04-03T16:26:27.6928015Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_sub_inplace_cuda_complex128 PASSED [3.3165s] [ 8%] 2024-04-03T16:26:27.6929390Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_tanh_outplace_cuda_complex128 PASSED [0.8743s] [ 8%] 2024-04-03T16:26:27.6930769Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_trunc_inplace_cuda_float64 PASSED [0.0939s] [ 8%] 2024-04-03T16:26:27.6932126Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_trunc_outplace_cuda_float64 PASSED [0.0884s] [ 8%] 2024-04-03T16:26:27.6933622Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_zero_inplace_cuda_complex128 PASSED [0.3990s] [ 8%] 2024-04-03T16:26:27.6935396Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_zero_outplace_cuda_complex128 SKIPPED [0.0024s] (out-of-place not implemented) [ 8%] 2024-04-03T16:26:27.6937292Z test_foreach.py::TestForeachCUDA::test_autodiff__foreach_zero_outplace_cuda_float64 SKIPPED [0.0022s] (out-of-place not implemented) [ 8%] 2024-04-03T16:26:27.6939437Z test_foreach.py::TestForeachCUDA::test_big_num_tensors__foreach_norm_cuda_float32 STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.6941318Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.6942929Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.6944367Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.6945740Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.6947199Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.6948631Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.6950024Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.6951457Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.6952399Z PASSED [0.9603s] [ 8%] 2024-04-03T16:26:27.6953874Z test_foreach.py::TestForeachCUDA::test_big_num_tensors__foreach_norm_cuda_float64 STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.6955733Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.6957189Z STAGE:2024-04-03 16:26:07 1666:1666 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.6958627Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.6960209Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.6961646Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.6963069Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.6964462Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.6965899Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.6966846Z PASSED [0.9197s] [ 8%] 2024-04-03T16:26:27.6968358Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_add_cuda_float64 STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.6970265Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.6971698Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.6972751Z ('RERUN', {'yellow': True}) [0.0106s] [ 8%] 2024-04-03T16:26:27.6974383Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_add_cuda_float64 STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.6976280Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.6977872Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.6978964Z ('RERUN', {'yellow': True}) [0.0099s] [ 8%] 2024-04-03T16:26:27.6980589Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_add_cuda_float64 STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.6982481Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.6983930Z STAGE:2024-04-03 16:26:08 1666:1666 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.6984870Z FAILED [0.0112s] [ 8%] 2024-04-03T16:26:27.6985143Z 2024-04-03T16:26:27.6985390Z ==================================== RERUNS ==================================== 2024-04-03T16:26:27.6986396Z ____ TestForeachCUDA.test_binary_op_float_inf_nan__foreach_add_cuda_float64 ____ 2024-04-03T16:26:27.6987197Z Traceback (most recent call last): 2024-04-03T16:26:27.6988138Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 539, in test_binary_op_float_inf_nan 2024-04-03T16:26:27.6989379Z self._binary_test(dtype, op, ref, inputs, True, False, alpha=None, scalar_self_arg=False) 2024-04-03T16:26:27.6990524Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 187, in _binary_test 2024-04-03T16:26:27.6991428Z actual = op(inputs, self.is_cuda, is_fastpath) 2024-04-03T16:26:27.6992280Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 66, in __call__ 2024-04-03T16:26:27.6993213Z assert mta_called == (expect_fastpath and (not zero_size)) 2024-04-03T16:26:27.6993853Z AssertionError 2024-04-03T16:26:27.6994490Z ____ TestForeachCUDA.test_binary_op_float_inf_nan__foreach_add_cuda_float64 ____ 2024-04-03T16:26:27.6995285Z Traceback (most recent call last): 2024-04-03T16:26:27.6996221Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 539, in test_binary_op_float_inf_nan 2024-04-03T16:26:27.6997474Z self._binary_test(dtype, op, ref, inputs, True, False, alpha=None, scalar_self_arg=False) 2024-04-03T16:26:27.6998617Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 187, in _binary_test 2024-04-03T16:26:27.6999674Z actual = op(inputs, self.is_cuda, is_fastpath) 2024-04-03T16:26:27.7000534Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 66, in __call__ 2024-04-03T16:26:27.7001462Z assert mta_called == (expect_fastpath and (not zero_size)) 2024-04-03T16:26:27.7002091Z AssertionError 2024-04-03T16:26:27.7002544Z =================================== FAILURES =================================== 2024-04-03T16:26:27.7003411Z ____ TestForeachCUDA.test_binary_op_float_inf_nan__foreach_add_cuda_float64 ____ 2024-04-03T16:26:27.7004198Z Traceback (most recent call last): 2024-04-03T16:26:27.7005106Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 539, in test_binary_op_float_inf_nan 2024-04-03T16:26:27.7006367Z self._binary_test(dtype, op, ref, inputs, True, False, alpha=None, scalar_self_arg=False) 2024-04-03T16:26:27.7007516Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 187, in _binary_test 2024-04-03T16:26:27.7008397Z actual = op(inputs, self.is_cuda, is_fastpath) 2024-04-03T16:26:27.7009253Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 66, in __call__ 2024-04-03T16:26:27.7010179Z assert mta_called == (expect_fastpath and (not zero_size)) 2024-04-03T16:26:27.7010793Z AssertionError 2024-04-03T16:26:27.7012038Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_foreach/test_foreach-9cbc8d00a0d5d7a6.xml - 2024-04-03T16:26:27.7013331Z =========================== short test summary info ============================ 2024-04-03T16:26:27.7014614Z FAILED [0.0112s] test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_add_cuda_float64 - AssertionError 2024-04-03T16:26:27.7015754Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-04-03T16:26:27.7016574Z ============== 1 failed, 94 passed, 5 skipped, 2 rerun in 59.75s =============== 2024-04-03T16:26:27.7017390Z Got exit code 1 2024-04-03T16:26:27.7017760Z Retrying... 2024-04-03T16:26:27.7018685Z Test results will be stored in test-reports/python-pytest/test_foreach/test_foreach-273a9009fcdeef67.xml 2024-04-03T16:26:27.7019750Z ============================= test session starts ============================== 2024-04-03T16:26:27.7020839Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.4.0 -- /opt/conda/envs/py_3.10/bin/python 2024-04-03T16:26:27.7021730Z cachedir: .pytest_cache 2024-04-03T16:26:27.7022849Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-04-03T16:26:27.7023932Z rootdir: /var/lib/jenkins/workspace 2024-04-03T16:26:27.7024411Z configfile: pytest.ini 2024-04-03T16:26:27.7025553Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-04-03T16:26:27.7026679Z collecting ... collected 2190 items / 99 deselected / 2091 selected 2024-04-03T16:26:27.7027408Z stepcurrent: skipping 99 already run items. 2024-04-03T16:26:27.7027974Z Running 1017 items in this shard 2024-04-03T16:26:27.7028305Z 2024-04-03T16:26:27.7029561Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_add_cuda_float64 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7031443Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7032894Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7034312Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7035710Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7037131Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7038070Z PASSED [1.0886s] [ 0%] 2024-04-03T16:26:27.7039793Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_clamp_max_cuda_float64 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7041710Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7043149Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7044568Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7045946Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7047383Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7048314Z PASSED [0.0521s] [ 0%] 2024-04-03T16:26:27.7049857Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_clamp_min_cuda_bfloat16 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7051776Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7053218Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7054639Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7056020Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7057440Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7058380Z PASSED [0.0210s] [ 0%] 2024-04-03T16:26:27.7060047Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_clamp_min_cuda_float16 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7062000Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7063441Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7064852Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7066234Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7067656Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7068587Z PASSED [0.0187s] [ 0%] 2024-04-03T16:26:27.7070308Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_clamp_min_cuda_float64 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7072248Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7073672Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7075093Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7076474Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7077899Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7078831Z PASSED [0.0193s] [ 0%] 2024-04-03T16:26:27.7080520Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_div_cuda_bfloat16 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7082446Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7083876Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7085294Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7086673Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7088100Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7089030Z PASSED [0.0218s] [ 0%] 2024-04-03T16:26:27.7090524Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_div_cuda_float64 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7092439Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7093872Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7095287Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7096667Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7098091Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7099024Z PASSED [0.0195s] [ 0%] 2024-04-03T16:26:27.7100542Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_maximum_cuda_float16 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7102454Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7103889Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7105439Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7106864Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7108288Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7109223Z PASSED [0.0169s] [ 0%] 2024-04-03T16:26:27.7110757Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_minimum_cuda_bfloat16 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7112736Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7114161Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7115739Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7117131Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7118606Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7119713Z PASSED [0.0201s] [ 0%] 2024-04-03T16:26:27.7121223Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_mul_cuda_bfloat16 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7123121Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7124552Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7125984Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7127371Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7128800Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7129742Z PASSED [0.0173s] [ 0%] 2024-04-03T16:26:27.7131238Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_mul_cuda_float64 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7133132Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7134555Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7135972Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7137364Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7138794Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7139731Z PASSED [0.0192s] [ 1%] 2024-04-03T16:26:27.7141235Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_pow_cuda_bfloat16 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7143130Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7144560Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7145974Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7147353Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7148785Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7149719Z PASSED [0.0570s] [ 1%] 2024-04-03T16:26:27.7151361Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_sub_cuda_bfloat16 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7153304Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7154740Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7156163Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7157548Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7158984Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7160239Z PASSED [0.0195s] [ 1%] 2024-04-03T16:26:27.7161763Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_sub_cuda_float16 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7163662Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7165094Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7166511Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7167894Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7169330Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7170251Z PASSED [0.0189s] [ 1%] 2024-04-03T16:26:27.7171758Z test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_sub_cuda_float32 STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7173664Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7175098Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7176514Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:314] Completed Stage: Warm Up 2024-04-03T16:26:27.7177898Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:320] Completed Stage: Collection 2024-04-03T16:26:27.7179337Z STAGE:2024-04-03 16:26:14 1715:1715 ActivityProfilerController.cpp:324] Completed Stage: Post Processing 2024-04-03T16:26:27.7180255Z PASSED [0.0169s] [ 1%] 2024-04-03T16:26:27.7183425Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_bfloat16 SKIPPED [0.0005s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/122790 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 1%] 2024-04-03T16:26:27.7189051Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_complex128 SKIPPED [0.0004s] (Test is disabled because an issue exists disabling it: https://github.com/pytorch/pytorch/issues/123003 for platform(s) linux. If you're seeing this on your local machine and would like to enable this test, please make sure CI is not set and you are not using the flag --import-disabled-tests.) [ 1%] 2024-04-03T16:26:27.7193110Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 [W403 16:26:14.849984432 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7194611Z 2024-04-03T16:26:27.7195411Z [W403 16:26:14.012475677 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7196505Z 2024-04-03T16:26:27.7197318Z [W403 16:26:14.019168861 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7198284Z 2024-04-03T16:26:27.7199241Z [W403 16:26:15.073309155 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7200228Z 2024-04-03T16:26:27.7200993Z [W403 16:26:15.091647263 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7201986Z 2024-04-03T16:26:27.7202749Z [W403 16:26:15.108791602 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7203899Z 2024-04-03T16:26:27.7204694Z [W403 16:26:15.109003803 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7205653Z 2024-04-03T16:26:27.7206449Z [W403 16:26:15.120259530 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7207411Z 2024-04-03T16:26:27.7208181Z [W403 16:26:15.126006979 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7209166Z 2024-04-03T16:26:27.7209938Z [W403 16:26:15.194386391 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7210919Z 2024-04-03T16:26:27.7211250Z ('RERUN', {'yellow': True}) [0.3678s] [ 1%] 2024-04-03T16:26:27.7212969Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 [W403 16:26:15.220966766 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7214482Z 2024-04-03T16:26:27.7215268Z [W403 16:26:15.221263746 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7216255Z 2024-04-03T16:26:27.7217027Z [W403 16:26:15.221932723 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7218012Z 2024-04-03T16:26:27.7218777Z [W403 16:26:15.222148873 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7219748Z 2024-04-03T16:26:27.7220532Z [W403 16:26:15.222533003 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7221494Z 2024-04-03T16:26:27.7222276Z [W403 16:26:15.222707893 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7223246Z 2024-04-03T16:26:27.7224019Z [W403 16:26:15.222862255 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7225001Z 2024-04-03T16:26:27.7225767Z [W403 16:26:15.223012433 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7226749Z 2024-04-03T16:26:27.7227511Z [W403 16:26:15.223722696 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7228469Z 2024-04-03T16:26:27.7229251Z [W403 16:26:15.225067179 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7230210Z 2024-04-03T16:26:27.7230483Z ('RERUN', {'yellow': True}) [0.0088s] [ 1%] 2024-04-03T16:26:27.7232209Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 [W403 16:26:15.231397054 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7233857Z 2024-04-03T16:26:27.7234659Z [W403 16:26:15.231674400 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7235621Z 2024-04-03T16:26:27.7236415Z [W403 16:26:15.232220980 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7237383Z 2024-04-03T16:26:27.7238168Z [W403 16:26:15.232431086 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7239312Z 2024-04-03T16:26:27.7240081Z [W403 16:26:15.232810670 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7241206Z 2024-04-03T16:26:27.7241967Z [W403 16:26:15.232983376 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7242951Z 2024-04-03T16:26:27.7243721Z [W403 16:26:15.233136592 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7244683Z 2024-04-03T16:26:27.7245477Z [W403 16:26:15.233285234 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7246437Z 2024-04-03T16:26:27.7247223Z [W403 16:26:15.233966646 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7248181Z 2024-04-03T16:26:27.7249038Z [W403 16:26:15.235271966 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.7249943Z 2024-04-03T16:26:27.7250092Z FAILED [0.0082s] [ 1%] 2024-04-03T16:26:27.7250359Z 2024-04-03T16:26:27.7250623Z ==================================== RERUNS ==================================== 2024-04-03T16:26:27.7251476Z __ TestForeachCUDA.test_binary_op_list_error_cases__foreach_add_cuda_float16 ___ 2024-04-03T16:26:27.7252280Z Traceback (most recent call last): 2024-04-03T16:26:27.7253224Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 438, in test_binary_op_list_error_cases 2024-04-03T16:26:27.7254185Z foreach_op(tensors1, tensors2) 2024-04-03T16:26:27.7255215Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.7256796Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.7257938Z C++ CapturedTraceback: 2024-04-03T16:26:27.7258907Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.7260749Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.7262264Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.7263301Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.7264279Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.7265560Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.7267043Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7268534Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7272659Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7276415Z #13 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7277830Z #14 at::native::foreach_tensor_add_list_kernel_slow(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7279618Z #15 at::native::foreach_tensor_add_list_kernel_cuda(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7281657Z #16 at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7287101Z #17 c10::impl::wrap_kernel_functor_unboxed_ > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7292282Z #18 at::_ops::_foreach_add_List::redispatch(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7294362Z #19 torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.7300219Z #20 c10::impl::wrap_kernel_functor_unboxed_ > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.7305562Z #21 at::_ops::_foreach_add_List::call(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7306969Z #22 torch::autograd::THPVariable__foreach_add(_object*, _object*, _object*) from python_torch_functions_1.cpp:0 2024-04-03T16:26:27.7307908Z #23 cfunction_call from :0 2024-04-03T16:26:27.7308390Z #24 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7308975Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7309516Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7310012Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.7310489Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7311026Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7311572Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.7312058Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7312594Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7313084Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.7313677Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7314243Z #35 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7314732Z #36 PyObject_Call from ??:0 2024-04-03T16:26:27.7315205Z #37 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7315727Z #38 method_vectorcall from :0 2024-04-03T16:26:27.7316192Z #39 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7316713Z #40 method_vectorcall from :0 2024-04-03T16:26:27.7317195Z #41 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7317697Z #42 method_vectorcall from :0 2024-04-03T16:26:27.7318173Z #43 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7318715Z #44 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7319421Z #45 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7319926Z #46 method_vectorcall from :0 2024-04-03T16:26:27.7320559Z #47 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7321059Z #48 method_vectorcall from :0 2024-04-03T16:26:27.7321543Z #49 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7322135Z #50 method_vectorcall from :0 2024-04-03T16:26:27.7322533Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7323051Z #52 method_vectorcall from :0 2024-04-03T16:26:27.7323512Z #53 PyObject_Call from ??:0 2024-04-03T16:26:27.7323971Z #54 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7324506Z #55 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7325120Z #56 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7325727Z #57 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7326225Z #58 slot_tp_call from :0 2024-04-03T16:26:27.7326703Z #59 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7327263Z #60 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7327795Z #61 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7328337Z #62 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7328849Z #63 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7329384Z #64 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7329919Z #65 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7330437Z #66 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7330957Z #67 method_vectorcall from :0 2024-04-03T16:26:27.7331442Z #68 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7331959Z #69 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7332563Z #70 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7333190Z #71 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7333673Z #72 slot_tp_call from :0 2024-04-03T16:26:27.7334105Z #73 PyObject_Call from ??:0 2024-04-03T16:26:27.7334579Z #74 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7335095Z #75 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7335623Z #76 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7336141Z #77 method_vectorcall from :0 2024-04-03T16:26:27.7336620Z #78 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7337152Z #79 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7337679Z #80 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7338201Z #81 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7338731Z #82 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7339267Z #83 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7339778Z #84 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7340309Z #85 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7340832Z #86 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7341347Z #87 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7341873Z #88 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7342395Z #89 method_vectorcall from :0 2024-04-03T16:26:27.7342859Z #90 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7343390Z #91 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7344003Z #92 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7344621Z #93 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7345125Z #94 slot_tp_call from :0 2024-04-03T16:26:27.7345605Z #95 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7346166Z #96 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7346812Z #97 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7347369Z #98 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7347885Z #99 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7348422Z #100 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7348950Z #101 method_vectorcall from :0 2024-04-03T16:26:27.7349424Z #102 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7349973Z #103 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7350594Z #104 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7351224Z #105 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7351710Z #106 slot_tp_call from :0 2024-04-03T16:26:27.7352202Z #107 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7352790Z #108 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7353446Z #109 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7353986Z #110 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7354527Z #111 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7355053Z #112 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7355594Z #113 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7356124Z #114 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7356655Z #115 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7357184Z #116 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7357709Z #117 method_vectorcall from :0 2024-04-03T16:26:27.7358186Z #118 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7358734Z #119 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7359543Z #120 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7360154Z #121 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7360656Z #122 slot_tp_call from :0 2024-04-03T16:26:27.7361146Z #123 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7361736Z #124 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7362282Z #125 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7362814Z #126 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7363354Z #127 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7363890Z #128 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7364408Z #129 _PyEval_Vector from :0 2024-04-03T16:26:27.7364845Z #130 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.7365315Z #131 run_eval_code_obj from :0 2024-04-03T16:26:27.7365759Z #132 run_mod from :0 2024-04-03T16:26:27.7366146Z #133 pyrun_file.cold from :0 2024-04-03T16:26:27.7366680Z #134 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.7367320Z #135 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.7367886Z #136 Py_RunMain.localalias from :0 2024-04-03T16:26:27.7368383Z #137 Py_BytesMain from ??:0 2024-04-03T16:26:27.7368838Z #138 __libc_start_main from ??:0 2024-04-03T16:26:27.7369292Z #139 _start from ??:0 2024-04-03T16:26:27.7369580Z 2024-04-03T16:26:27.7369599Z 2024-04-03T16:26:27.7369962Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.7370524Z 2024-04-03T16:26:27.7371272Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.7372839Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.7373990Z C++ CapturedTraceback: 2024-04-03T16:26:27.7374972Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.7376801Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.7378294Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.7379345Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.7380349Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.7381742Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.7383253Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7384760Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7388748Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7392753Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7394572Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.7399437Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.7403636Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7405290Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.7406829Z #18 cfunction_call from :0 2024-04-03T16:26:27.7407683Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7408680Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7409605Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7410565Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7411639Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7412526Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.7413331Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7414318Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7415195Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.7416003Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7417068Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7418025Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.7418988Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7419991Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7420912Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.7421800Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7422786Z #35 method_vectorcall from :0 2024-04-03T16:26:27.7423682Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7424676Z #37 method_vectorcall from :0 2024-04-03T16:26:27.7425626Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7426656Z #39 method_vectorcall from :0 2024-04-03T16:26:27.7427529Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7428597Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7429639Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7430562Z #43 method_vectorcall from :0 2024-04-03T16:26:27.7431440Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7432429Z #45 method_vectorcall from :0 2024-04-03T16:26:27.7433588Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7434674Z #47 method_vectorcall from :0 2024-04-03T16:26:27.7435651Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7436675Z #49 method_vectorcall from :0 2024-04-03T16:26:27.7437579Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.7438512Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7439892Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7441107Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7442361Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7443336Z #55 slot_tp_call from :0 2024-04-03T16:26:27.7444247Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7445414Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7446720Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7447779Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7448778Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7449759Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7450810Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7451771Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7452708Z #64 method_vectorcall from :0 2024-04-03T16:26:27.7453632Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7454720Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7455897Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7457179Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7458150Z #69 slot_tp_call from :0 2024-04-03T16:26:27.7458974Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.7459897Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7460914Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7461871Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7462871Z #74 method_vectorcall from :0 2024-04-03T16:26:27.7463770Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7464751Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7465793Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7466827Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7467821Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7468853Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7469917Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7470985Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7471978Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7473078Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7474186Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7475254Z #86 method_vectorcall from :0 2024-04-03T16:26:27.7476258Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7477372Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7478635Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7480283Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7481324Z #91 slot_tp_call from :0 2024-04-03T16:26:27.7482298Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7483520Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7484667Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7485775Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7486909Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7488029Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7489111Z #98 method_vectorcall from :0 2024-04-03T16:26:27.7490119Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7491258Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7492527Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7493871Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7494926Z #103 slot_tp_call from :0 2024-04-03T16:26:27.7495940Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7497158Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7498291Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7499390Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7500783Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7501936Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7503065Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7504160Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7505194Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7506188Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7507201Z #114 method_vectorcall from :0 2024-04-03T16:26:27.7508186Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7509245Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7510439Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7511742Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7512981Z #119 slot_tp_call from :0 2024-04-03T16:26:27.7513894Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7515017Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7516029Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7516886Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7517446Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7518073Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7519385Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.7520306Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.7521192Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.7522055Z #129 run_mod from :0 2024-04-03T16:26:27.7522854Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.7523872Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.7525125Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.7526300Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.7527259Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.7528179Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.7529096Z #136 _start from ??:0 2024-04-03T16:26:27.7529608Z 2024-04-03T16:26:27.7529636Z 2024-04-03T16:26:27.7530343Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.7531460Z 2024-04-03T16:26:27.7531826Z Traceback (most recent call last): 2024-04-03T16:26:27.7533598Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 440, in test_binary_op_list_error_cases 2024-04-03T16:26:27.7535709Z with self.assertRaisesRegex(type(e), re.escape(str(e))): 2024-04-03T16:26:27.7536764Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 239, in __exit__ 2024-04-03T16:26:27.7538575Z self._raiseFailure('"{}" does not match "{}"'.format( 2024-04-03T16:26:27.7540409Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 163, in _raiseFailure 2024-04-03T16:26:27.7541431Z raise self.test_case.failureException(msg) 2024-04-03T16:26:27.7542783Z AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.7544699Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.7545955Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.7547077Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7549132Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7550849Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7552011Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7553148Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7554624Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7556560Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7558322Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.7563305Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.7567682Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7569284Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7571076Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7573159Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.7579054Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.7584649Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7586925Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.7593238Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.7599136Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7601181Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.7602252Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.7602799Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.7603447Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7604055Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7604605Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.7605155Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7605761Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7606307Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.7606853Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7607452Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7608120Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.7608667Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7609267Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7609812Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.7610365Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7610936Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7611514Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7612088Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7612640Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7613197Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7613743Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7614337Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7614924Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7615489Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7616048Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7616598Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7617144Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7617718Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7618262Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7618810Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7619314Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.7619860Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7620443Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7621114Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.7621792Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.7622326Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.7622961Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.7623506Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7624102Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7624703Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7625305Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7625890Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7626492Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7627090Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7627638Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7628184Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7628788Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7629437Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.7630116Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.7630670Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.7631123Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.7631670Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7632264Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7632861Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7633434Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7633979Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7634650Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7635270Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7635862Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7636442Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7637040Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7637637Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7638212Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7638817Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7639814Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7640406Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7640975Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7641680Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7642259Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7642938Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.7643616Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.7644149Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.7644688Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.7645324Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7645922Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7646509Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7647105Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7647718Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7648288Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7648848Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7649454Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7650134Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.7650824Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.7651392Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.7651922Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.7652574Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7653185Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7653782Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7654392Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7655003Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7655598Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7656202Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7656807Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7657394Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7657983Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.7658545Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7659135Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7659824Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.7660515Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.7661058Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.7661609Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.7662267Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7662856Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7663467Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7664078Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.7664669Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.7665242Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.7665744Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.7666263Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.7666754Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.7667201Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.7704864Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.7705656Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.7706309Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.7706855Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.7707388Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.7707931Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.7709008Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.7710649Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.7711944Z C++ CapturedTraceback: 2024-04-03T16:26:27.7712934Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.7714953Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.7716497Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.7717555Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.7718559Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.7720369Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.7721881Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7723388Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7727405Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7737516Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7739354Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.7744047Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.7748231Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7749451Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.7750431Z #18 cfunction_call from :0 2024-04-03T16:26:27.7750914Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7751495Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7752016Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7752530Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7753038Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7753691Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.7754174Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7754683Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7755163Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.7755606Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7756115Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7756597Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.7757047Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7757570Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7758064Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.7758548Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7759678Z #35 method_vectorcall from :0 2024-04-03T16:26:27.7760285Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7760815Z #37 method_vectorcall from :0 2024-04-03T16:26:27.7761305Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7761813Z #39 method_vectorcall from :0 2024-04-03T16:26:27.7762312Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7762861Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7763380Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7763907Z #43 method_vectorcall from :0 2024-04-03T16:26:27.7764396Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7764905Z #45 method_vectorcall from :0 2024-04-03T16:26:27.7765396Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7765923Z #47 method_vectorcall from :0 2024-04-03T16:26:27.7766390Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7766920Z #49 method_vectorcall from :0 2024-04-03T16:26:27.7767389Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.7767851Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7768410Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7769033Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7769653Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7770165Z #55 slot_tp_call from :0 2024-04-03T16:26:27.7770663Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7771238Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7771783Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7772323Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7772851Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7773391Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7773933Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7774451Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7774977Z #64 method_vectorcall from :0 2024-04-03T16:26:27.7775465Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7775987Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7776598Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7777245Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7777733Z #69 slot_tp_call from :0 2024-04-03T16:26:27.7778171Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.7778669Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7779201Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7779738Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7780271Z #74 method_vectorcall from :0 2024-04-03T16:26:27.7780747Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7781289Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7781824Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7782346Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7782884Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7783426Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7783945Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7784485Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7785028Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7785552Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7786091Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7786620Z #86 method_vectorcall from :0 2024-04-03T16:26:27.7787221Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7787781Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7788396Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7789006Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7789509Z #91 slot_tp_call from :0 2024-04-03T16:26:27.7789997Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7790569Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7791113Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7791651Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7792171Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7792710Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7793388Z #98 method_vectorcall from :0 2024-04-03T16:26:27.7793862Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7794410Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7795041Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7795677Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7796194Z #103 slot_tp_call from :0 2024-04-03T16:26:27.7796693Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7797272Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7797827Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7798371Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7798904Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7799681Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7800225Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7800747Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7801302Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7801843Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7802375Z #114 method_vectorcall from :0 2024-04-03T16:26:27.7802881Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7803437Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7804071Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7804702Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7805212Z #119 slot_tp_call from :0 2024-04-03T16:26:27.7805713Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7806293Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7806846Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7807392Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7807927Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7808464Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7808988Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.7809433Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.7809909Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.7810373Z #129 run_mod from :0 2024-04-03T16:26:27.7810810Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.7811363Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.7812026Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.7812599Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.7813107Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.7813574Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.7814028Z #136 _start from ??:0 2024-04-03T16:26:27.7814412Z " 2024-04-03T16:26:27.7814593Z 2024-04-03T16:26:27.7814947Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.7816082Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.7816781Z 2024-04-03T16:26:27.7817193Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.7818227Z __ TestForeachCUDA.test_binary_op_list_error_cases__foreach_add_cuda_float16 ___ 2024-04-03T16:26:27.7819020Z Traceback (most recent call last): 2024-04-03T16:26:27.7819971Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 438, in test_binary_op_list_error_cases 2024-04-03T16:26:27.7820942Z foreach_op(tensors1, tensors2) 2024-04-03T16:26:27.7822140Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.7823765Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.7824918Z C++ CapturedTraceback: 2024-04-03T16:26:27.7825918Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.7827735Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.7829255Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.7830435Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.7831426Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.7832719Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.7834218Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7835738Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7839911Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7843672Z #13 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7845095Z #14 at::native::foreach_tensor_add_list_kernel_slow(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7846720Z #15 at::native::foreach_tensor_add_list_kernel_cuda(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7848643Z #16 at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7854094Z #17 c10::impl::wrap_kernel_functor_unboxed_ > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7859298Z #18 at::_ops::_foreach_add_List::redispatch(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7861362Z #19 torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.7867363Z #20 c10::impl::wrap_kernel_functor_unboxed_ > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.7872749Z #21 at::_ops::_foreach_add_List::call(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7874278Z #22 torch::autograd::THPVariable__foreach_add(_object*, _object*, _object*) from python_torch_functions_1.cpp:0 2024-04-03T16:26:27.7875223Z #23 cfunction_call from :0 2024-04-03T16:26:27.7875706Z #24 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7876300Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7876840Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7877342Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.7877820Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7878362Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7878864Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.7879573Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7880111Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7880609Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.7881092Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7881640Z #35 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7882147Z #36 PyObject_Call from ??:0 2024-04-03T16:26:27.7882629Z #37 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7883161Z #38 method_vectorcall from :0 2024-04-03T16:26:27.7883644Z #39 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7884173Z #40 method_vectorcall from :0 2024-04-03T16:26:27.7884666Z #41 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7885173Z #42 method_vectorcall from :0 2024-04-03T16:26:27.7885667Z #43 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7886205Z #44 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7886723Z #45 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7887246Z #46 method_vectorcall from :0 2024-04-03T16:26:27.7887736Z #47 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7888263Z #48 method_vectorcall from :0 2024-04-03T16:26:27.7888733Z #49 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7889259Z #50 method_vectorcall from :0 2024-04-03T16:26:27.7889752Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7890269Z #52 method_vectorcall from :0 2024-04-03T16:26:27.7890720Z #53 PyObject_Call from ??:0 2024-04-03T16:26:27.7891182Z #54 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7891720Z #55 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7892332Z #56 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7892956Z #57 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7893443Z #58 slot_tp_call from :0 2024-04-03T16:26:27.7893897Z #59 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7894447Z #60 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7894946Z #61 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7895444Z #62 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7895952Z #63 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7896443Z #64 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7896946Z #65 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7897444Z #66 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7897942Z #67 method_vectorcall from :0 2024-04-03T16:26:27.7898398Z #68 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7898897Z #69 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7899589Z #70 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7900217Z #71 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7900684Z #72 slot_tp_call from :0 2024-04-03T16:26:27.7901077Z #73 PyObject_Call from ??:0 2024-04-03T16:26:27.7901522Z #74 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7902025Z #75 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7902521Z #76 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7903012Z #77 method_vectorcall from :0 2024-04-03T16:26:27.7903468Z #78 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7903967Z #79 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7904464Z #80 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7904961Z #81 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7905453Z #82 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7906086Z #83 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7906585Z #84 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7907076Z #85 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7907577Z #86 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7908085Z #87 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7908579Z #88 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7909067Z #89 method_vectorcall from :0 2024-04-03T16:26:27.7909524Z #90 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7910018Z #91 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7910593Z #92 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7911243Z #93 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7911706Z #94 slot_tp_call from :0 2024-04-03T16:26:27.7912154Z #95 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7912703Z #96 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7913198Z #97 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7913709Z #98 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7914217Z #99 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7914717Z #100 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7915225Z #101 method_vectorcall from :0 2024-04-03T16:26:27.7915686Z #102 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7916194Z #103 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7916783Z #104 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7917389Z #105 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7917857Z #106 slot_tp_call from :0 2024-04-03T16:26:27.7918324Z #107 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7918886Z #108 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7919574Z #109 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7920085Z #110 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7920597Z #111 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7921093Z #112 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7921614Z #113 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7922113Z #114 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7922624Z #115 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7923130Z #116 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7923625Z #117 method_vectorcall from :0 2024-04-03T16:26:27.7924081Z #118 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7924596Z #119 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7925178Z #120 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7925782Z #121 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7926252Z #122 slot_tp_call from :0 2024-04-03T16:26:27.7926715Z #123 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7927269Z #124 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7927785Z #125 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7927958Z #126 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7928133Z #127 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7928320Z #128 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7928462Z #129 _PyEval_Vector from :0 2024-04-03T16:26:27.7928623Z #130 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.7928766Z #131 run_eval_code_obj from :0 2024-04-03T16:26:27.7929023Z #132 run_mod from :0 2024-04-03T16:26:27.7929205Z #133 pyrun_file.cold from :0 2024-04-03T16:26:27.7929441Z #134 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.7929660Z #135 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.7929827Z #136 Py_RunMain.localalias from :0 2024-04-03T16:26:27.7929976Z #137 Py_BytesMain from ??:0 2024-04-03T16:26:27.7930132Z #138 __libc_start_main from ??:0 2024-04-03T16:26:27.7930274Z #139 _start from ??:0 2024-04-03T16:26:27.7930295Z 2024-04-03T16:26:27.7930314Z 2024-04-03T16:26:27.7930685Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.7930706Z 2024-04-03T16:26:27.7931456Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.7932413Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.7932595Z C++ CapturedTraceback: 2024-04-03T16:26:27.7933321Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.7934253Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.7934673Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.7935121Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.7935490Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.7936235Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.7936822Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7937597Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7940662Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.7941354Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7942336Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.7945876Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.7946385Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.7946930Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.7947088Z #18 cfunction_call from :0 2024-04-03T16:26:27.7947382Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7947576Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7947750Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7947924Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7948097Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7948258Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.7948440Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7948631Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7948790Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.7948972Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7949163Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7949324Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.7949605Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7949798Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7949958Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.7950137Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7950325Z #35 method_vectorcall from :0 2024-04-03T16:26:27.7950508Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7950669Z #37 method_vectorcall from :0 2024-04-03T16:26:27.7950867Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7951026Z #39 method_vectorcall from :0 2024-04-03T16:26:27.7951205Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7951398Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7951578Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7951739Z #43 method_vectorcall from :0 2024-04-03T16:26:27.7951938Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7952097Z #45 method_vectorcall from :0 2024-04-03T16:26:27.7952297Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7952467Z #47 method_vectorcall from :0 2024-04-03T16:26:27.7952647Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7952825Z #49 method_vectorcall from :0 2024-04-03T16:26:27.7952983Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.7953170Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7953367Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7953631Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7953806Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7953976Z #55 slot_tp_call from :0 2024-04-03T16:26:27.7954189Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7954379Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7954569Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7954749Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7954925Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7955131Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7955305Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7955493Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7955674Z #64 method_vectorcall from :0 2024-04-03T16:26:27.7955853Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7956052Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7956314Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7956488Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7956656Z #69 slot_tp_call from :0 2024-04-03T16:26:27.7956817Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.7956999Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7957190Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7957371Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7957531Z #74 method_vectorcall from :0 2024-04-03T16:26:27.7957730Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7957904Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7958084Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7958275Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7958463Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7958638Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7958838Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7959292Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7959519Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7959712Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7959892Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7960073Z #86 method_vectorcall from :0 2024-04-03T16:26:27.7960252Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7960425Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7960702Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7960876Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7961027Z #91 slot_tp_call from :0 2024-04-03T16:26:27.7961259Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7961440Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7961746Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7961946Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7962120Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7962305Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7962485Z #98 method_vectorcall from :0 2024-04-03T16:26:27.7962665Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7962846Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7963136Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7963317Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7963470Z #103 slot_tp_call from :0 2024-04-03T16:26:27.7963706Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7963893Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7964095Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7964280Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7964457Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7964668Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7964846Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7965030Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7965231Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7965416Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7965581Z #114 method_vectorcall from :0 2024-04-03T16:26:27.7965781Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7965960Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7966233Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.7966430Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.7966583Z #119 slot_tp_call from :0 2024-04-03T16:26:27.7966801Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.7967005Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7967182Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7967365Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7967573Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.7967758Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.7967934Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.7968103Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.7968271Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.7968435Z #129 run_mod from :0 2024-04-03T16:26:27.7968597Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.7968843Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.7969084Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.7969260Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.7969421Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.7969604Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.7969753Z #136 _start from ??:0 2024-04-03T16:26:27.7969775Z 2024-04-03T16:26:27.7969795Z 2024-04-03T16:26:27.7970176Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.7970196Z 2024-04-03T16:26:27.7970396Z Traceback (most recent call last): 2024-04-03T16:26:27.7971016Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 440, in test_binary_op_list_error_cases 2024-04-03T16:26:27.7971344Z with self.assertRaisesRegex(type(e), re.escape(str(e))): 2024-04-03T16:26:27.7971923Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 239, in __exit__ 2024-04-03T16:26:27.7972297Z self._raiseFailure('"{}" does not match "{}"'.format( 2024-04-03T16:26:27.7972835Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 163, in _raiseFailure 2024-04-03T16:26:27.7973066Z raise self.test_case.failureException(msg) 2024-04-03T16:26:27.7973979Z AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.7974924Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.7975094Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.7976029Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7977103Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7977563Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7978102Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7978529Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7979366Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7980065Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7980951Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.7984423Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.7985030Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7985855Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7986643Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7987784Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.7992463Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.7993408Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.7994614Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.7999744Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.8000631Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8001331Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.8001511Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.8001754Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8001993Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8002202Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8002380Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8002618Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8002820Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8003009Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8003223Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8003421Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8003606Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8003816Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8004011Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8004197Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8004419Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8004589Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8004817Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8004991Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8005198Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8005385Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8005597Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8005795Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8006022Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8006193Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8006422Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8006594Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8006802Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8006992Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8007198Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8007376Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8007560Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8007767Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8008080Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8008400Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8008600Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8008761Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8009010Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8009220Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8009417Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8009644Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8009840Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8010047Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8010264Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8010577Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8010811Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8011022Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8011225Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8011525Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8011723Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8011886Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8012075Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8012287Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8012485Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8012711Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8012882Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8013092Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8013310Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8013532Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8013729Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8013955Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8014157Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8014385Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8014581Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8014791Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8015008Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8015215Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8015388Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8015615Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8015812Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8016094Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8016312Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8016481Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8016714Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8016945Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8017146Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8017379Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8017576Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8017799Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8017992Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8018208Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8018414Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8018721Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8018926Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8019089Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8019347Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8019572Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8019776Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8020012Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8020325Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8020557Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8020779Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8020992Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8021213Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8021431Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8021607Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8021846Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8022051Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8022339Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8022659Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8022824Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8023061Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8023298Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8023499Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8023712Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8023935Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8024148Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8024315Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.8024514Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.8024683Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.8024856Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.8025027Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.8025290Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8025542Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8025742Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.8025911Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.8026107Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.8026270Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.8027024Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8027854Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8028017Z C++ CapturedTraceback: 2024-04-03T16:26:27.8028740Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8029692Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8030102Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8030568Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8030937Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8031668Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8032261Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8033011Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8036253Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8036960Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8037934Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8041667Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8042357Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8042908Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.8043063Z #18 cfunction_call from :0 2024-04-03T16:26:27.8043274Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8043478Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8043654Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8043853Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8044026Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8044185Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.8044392Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8044564Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8044721Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8044922Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8045098Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8045254Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8045449Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8045620Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8045776Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8045974Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8046136Z #35 method_vectorcall from :0 2024-04-03T16:26:27.8046313Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8046489Z #37 method_vectorcall from :0 2024-04-03T16:26:27.8046664Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8046819Z #39 method_vectorcall from :0 2024-04-03T16:26:27.8047014Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8047199Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8047376Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8047552Z #43 method_vectorcall from :0 2024-04-03T16:26:27.8047727Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8047906Z #45 method_vectorcall from :0 2024-04-03T16:26:27.8048085Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8048241Z #47 method_vectorcall from :0 2024-04-03T16:26:27.8048432Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8048587Z #49 method_vectorcall from :0 2024-04-03T16:26:27.8048740Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.8048933Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8049103Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8049364Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8049555Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8049702Z #55 slot_tp_call from :0 2024-04-03T16:26:27.8049910Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8050114Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8050284Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8050463Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8050760Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8050955Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8051124Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8051321Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8051477Z #64 method_vectorcall from :0 2024-04-03T16:26:27.8051668Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8051838Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8052094Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8052280Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8052427Z #69 slot_tp_call from :0 2024-04-03T16:26:27.8052583Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.8052776Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8053049Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8053226Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8053403Z #74 method_vectorcall from :0 2024-04-03T16:26:27.8053581Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8053756Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8053949Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8054118Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8054294Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8054482Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8054659Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8054828Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8055024Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8055195Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8055388Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8055546Z #86 method_vectorcall from :0 2024-04-03T16:26:27.8055722Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8055918Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8056174Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8056343Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8056511Z #91 slot_tp_call from :0 2024-04-03T16:26:27.8056720Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8056898Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8057085Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8057261Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8057429Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8057623Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8057778Z #98 method_vectorcall from :0 2024-04-03T16:26:27.8057953Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8058148Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8058419Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8058593Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8058770Z #103 slot_tp_call from :0 2024-04-03T16:26:27.8058988Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8059188Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8059370Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8059554Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8059744Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8059924Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8060096Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8060289Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8060462Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8060641Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8060817Z #114 method_vectorcall from :0 2024-04-03T16:26:27.8060994Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8061169Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8061451Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8061637Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8061787Z #119 slot_tp_call from :0 2024-04-03T16:26:27.8062018Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8062286Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8062477Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8062674Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8062846Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8063042Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8063195Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.8063356Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8063528Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.8063668Z #129 run_mod from :0 2024-04-03T16:26:27.8063824Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.8064083Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8064301Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8064474Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8064753Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.8064913Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.8065054Z #136 _start from ??:0 2024-04-03T16:26:27.8065195Z " 2024-04-03T16:26:27.8065218Z 2024-04-03T16:26:27.8065560Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.8066166Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.8066187Z 2024-04-03T16:26:27.8066617Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.8066863Z =================================== FAILURES =================================== 2024-04-03T16:26:27.8067306Z __ TestForeachCUDA.test_binary_op_list_error_cases__foreach_add_cuda_float16 ___ 2024-04-03T16:26:27.8067484Z Traceback (most recent call last): 2024-04-03T16:26:27.8068087Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 438, in test_binary_op_list_error_cases 2024-04-03T16:26:27.8068286Z foreach_op(tensors1, tensors2) 2024-04-03T16:26:27.8068995Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8069801Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8069978Z C++ CapturedTraceback: 2024-04-03T16:26:27.8070693Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8071637Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8072030Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8072470Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8072852Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8073585Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8074161Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8074923Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8077976Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8078508Z #13 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8079541Z #14 at::native::foreach_tensor_add_list_kernel_slow(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8080316Z #15 at::native::foreach_tensor_add_list_kernel_cuda(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8081318Z #16 at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8085584Z #17 c10::impl::wrap_kernel_functor_unboxed_ > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8086507Z #18 at::_ops::_foreach_add_List::redispatch(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8087616Z #19 torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.8092196Z #20 c10::impl::wrap_kernel_functor_unboxed_ > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.8092848Z #21 at::_ops::_foreach_add_List::call(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8093435Z #22 torch::autograd::THPVariable__foreach_add(_object*, _object*, _object*) from python_torch_functions_1.cpp:0 2024-04-03T16:26:27.8093597Z #23 cfunction_call from :0 2024-04-03T16:26:27.8093827Z #24 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8094012Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8094207Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8094368Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8094549Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8094738Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8094895Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8095075Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8095262Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8095416Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8095594Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8095784Z #35 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8095938Z #36 PyObject_Call from ??:0 2024-04-03T16:26:27.8096115Z #37 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8096291Z #38 method_vectorcall from :0 2024-04-03T16:26:27.8096475Z #39 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8096715Z #40 method_vectorcall from :0 2024-04-03T16:26:27.8096907Z #41 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8097086Z #42 method_vectorcall from :0 2024-04-03T16:26:27.8097416Z #43 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8097487Z #44 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8097646Z #45 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8097820Z #46 method_vectorcall from :0 2024-04-03T16:26:27.8097997Z #47 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8098152Z #48 method_vectorcall from :0 2024-04-03T16:26:27.8098345Z #49 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8098504Z #50 method_vectorcall from :0 2024-04-03T16:26:27.8098680Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8098854Z #52 method_vectorcall from :0 2024-04-03T16:26:27.8099009Z #53 PyObject_Call from ??:0 2024-04-03T16:26:27.8099184Z #54 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8099471Z #55 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8099733Z #56 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8099904Z #57 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8100077Z #58 slot_tp_call from :0 2024-04-03T16:26:27.8100285Z #59 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8100463Z #60 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8100653Z #61 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8100832Z #62 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8101000Z #63 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8101197Z #64 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8101369Z #65 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8101545Z #66 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8101724Z #67 method_vectorcall from :0 2024-04-03T16:26:27.8101900Z #68 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8102087Z #69 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8102353Z #70 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8102526Z #71 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8102691Z #72 slot_tp_call from :0 2024-04-03T16:26:27.8102847Z #73 PyObject_Call from ??:0 2024-04-03T16:26:27.8103031Z #74 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8103218Z #75 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8103397Z #76 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8103554Z #77 method_vectorcall from :0 2024-04-03T16:26:27.8103747Z #78 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8103918Z #79 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8104094Z #80 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8104284Z #81 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8104459Z #82 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8104628Z #83 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8104823Z #84 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8104993Z #85 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8105178Z #86 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8105368Z #87 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8105542Z #88 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8105726Z #89 method_vectorcall from :0 2024-04-03T16:26:27.8105905Z #90 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8106074Z #91 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8106345Z #92 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8106515Z #93 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8106662Z #94 slot_tp_call from :0 2024-04-03T16:26:27.8106885Z #95 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8107062Z #96 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8107232Z #97 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8107428Z #98 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8107596Z #99 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8107779Z #100 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8107963Z #101 method_vectorcall from :0 2024-04-03T16:26:27.8108145Z #102 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8108322Z #103 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8108695Z #104 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8108885Z #105 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8109034Z #106 slot_tp_call from :0 2024-04-03T16:26:27.8109267Z #107 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8109446Z #108 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8109639Z #109 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8109820Z #110 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8109994Z #111 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8110188Z #112 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8110362Z #113 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8110541Z #114 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8110734Z #115 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8111066Z #116 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8111226Z #117 method_vectorcall from :0 2024-04-03T16:26:27.8111422Z #118 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8111604Z #119 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8111870Z #120 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8112061Z #121 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8112210Z #122 slot_tp_call from :0 2024-04-03T16:26:27.8112424Z #123 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8112622Z #124 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8112798Z #125 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8112977Z #126 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8113169Z #127 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8113347Z #128 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8113518Z #129 _PyEval_Vector from :0 2024-04-03T16:26:27.8113681Z #130 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8113842Z #131 run_eval_code_obj from :0 2024-04-03T16:26:27.8113998Z #132 run_mod from :0 2024-04-03T16:26:27.8114156Z #133 pyrun_file.cold from :0 2024-04-03T16:26:27.8114395Z #134 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8114637Z #135 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8114808Z #136 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8114962Z #137 Py_BytesMain from ??:0 2024-04-03T16:26:27.8115139Z #138 __libc_start_main from ??:0 2024-04-03T16:26:27.8115286Z #139 _start from ??:0 2024-04-03T16:26:27.8115307Z 2024-04-03T16:26:27.8115326Z 2024-04-03T16:26:27.8115695Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.8115716Z 2024-04-03T16:26:27.8116457Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8117259Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8117440Z C++ CapturedTraceback: 2024-04-03T16:26:27.8118154Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8119271Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8119692Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8120134Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8120516Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8121239Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8121808Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8122576Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8125758Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8126482Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8127451Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8131134Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8131640Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8132191Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.8132347Z #18 cfunction_call from :0 2024-04-03T16:26:27.8132565Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8132765Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8132940Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8133126Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8133318Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8133476Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.8133656Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8133842Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8133998Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8134175Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8134364Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8134524Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8134699Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8134885Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8135040Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8135218Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8135403Z #35 method_vectorcall from :0 2024-04-03T16:26:27.8135580Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8135735Z #37 method_vectorcall from :0 2024-04-03T16:26:27.8135937Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8136093Z #39 method_vectorcall from :0 2024-04-03T16:26:27.8136285Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8136457Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8136632Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8136805Z #43 method_vectorcall from :0 2024-04-03T16:26:27.8136983Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8137141Z #45 method_vectorcall from :0 2024-04-03T16:26:27.8137334Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8137490Z #47 method_vectorcall from :0 2024-04-03T16:26:27.8137766Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8137941Z #49 method_vectorcall from :0 2024-04-03T16:26:27.8138000Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.8138264Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8138406Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8138627Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8138888Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8139072Z #55 slot_tp_call from :0 2024-04-03T16:26:27.8139280Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8139458Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8139649Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8139825Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8140012Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8140190Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8140359Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8140550Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8140707Z #64 method_vectorcall from :0 2024-04-03T16:26:27.8140884Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8141187Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8141445Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8141613Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8141783Z #69 slot_tp_call from :0 2024-04-03T16:26:27.8141939Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.8142118Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8142306Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8142483Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8142638Z #74 method_vectorcall from :0 2024-04-03T16:26:27.8142831Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8143002Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8143179Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8143365Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8143542Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8143711Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8143907Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8144085Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8144279Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8144449Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8144630Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8144806Z #86 method_vectorcall from :0 2024-04-03T16:26:27.8144983Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8145152Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8145427Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8145598Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8145743Z #91 slot_tp_call from :0 2024-04-03T16:26:27.8145967Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8146147Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8146316Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8146514Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8146684Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8146871Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8147045Z #98 method_vectorcall from :0 2024-04-03T16:26:27.8147224Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8147409Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8147699Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8147877Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8148043Z #103 slot_tp_call from :0 2024-04-03T16:26:27.8148260Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8148445Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8148637Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8148815Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8148988Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8149185Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8149359Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8149540Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8149741Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8149921Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8150080Z #114 method_vectorcall from :0 2024-04-03T16:26:27.8150366Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8150557Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8150824Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8151018Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8151166Z #119 slot_tp_call from :0 2024-04-03T16:26:27.8151397Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8151579Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8151753Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8151947Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8152121Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8152301Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8152470Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.8152736Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8152894Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.8153051Z #129 run_mod from :0 2024-04-03T16:26:27.8153207Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.8153453Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8153690Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8153863Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8154017Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.8154196Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.8154339Z #136 _start from ??:0 2024-04-03T16:26:27.8154360Z 2024-04-03T16:26:27.8154379Z 2024-04-03T16:26:27.8154769Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.8154790Z 2024-04-03T16:26:27.8154963Z Traceback (most recent call last): 2024-04-03T16:26:27.8155571Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 440, in test_binary_op_list_error_cases 2024-04-03T16:26:27.8155891Z with self.assertRaisesRegex(type(e), re.escape(str(e))): 2024-04-03T16:26:27.8156381Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 239, in __exit__ 2024-04-03T16:26:27.8156731Z self._raiseFailure('"{}" does not match "{}"'.format( 2024-04-03T16:26:27.8157268Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 163, in _raiseFailure 2024-04-03T16:26:27.8157496Z raise self.test_case.failureException(msg) 2024-04-03T16:26:27.8158395Z AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.8159510Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.8159688Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.8160510Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8161587Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8162045Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8162580Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8162998Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8163842Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8164518Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8165384Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8168979Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8169611Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8170421Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8171311Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8172453Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8177039Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8177962Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8179167Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.8184107Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.8184819Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8185520Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.8185689Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.8185926Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8186169Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8186371Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8186559Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8186865Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8187079Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8187264Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8187471Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8187661Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8187843Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8188049Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8188240Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8188423Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8188630Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8188795Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8189123Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8189286Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8189491Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8189676Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8189886Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8190098Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8190300Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8190466Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8190688Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8190853Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8191055Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8191239Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8191444Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8191611Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8191794Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8191998Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8192204Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8192502Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8192702Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8192859Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8193106Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8193311Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8193523Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8193728Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8193920Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8194140Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8194331Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8194538Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8194723Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8194926Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8195130Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8195424Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8195623Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8195784Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8195967Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8196172Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8196364Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8196588Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8196752Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8196974Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8197169Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8197375Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8197583Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8197787Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8197988Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8198208Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8198400Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8198711Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8198937Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8199317Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8199486Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8199781Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8200007Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8200272Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8200437Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8200545Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8200790Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8200998Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8201334Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8201556Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8201746Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8201971Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8202158Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8202375Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8202578Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8202881Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8203082Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8203240Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8203494Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8203709Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8203908Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8204148Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8204349Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8204574Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8204778Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8204989Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8205203Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8205412Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8205583Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8205812Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8206011Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8206296Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8206518Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8206680Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8206911Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8207148Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8207348Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8207558Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8207780Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8207992Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8208173Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.8208350Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.8208516Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.8208685Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.8208850Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.8209106Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8209354Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8209544Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.8209708Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.8209901Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.8210063Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.8210849Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8211810Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8211992Z C++ CapturedTraceback: 2024-04-03T16:26:27.8212732Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8213661Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8214057Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8214520Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8214985Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8215709Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8216305Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8217056Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8220133Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8220818Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8221806Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8225320Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8225848Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8226390Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.8226548Z #18 cfunction_call from :0 2024-04-03T16:26:27.8226780Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8226966Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8227140Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8227339Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8227513Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8227687Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.8227867Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8228037Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8228209Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8228397Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8228567Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8228739Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8228917Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8229177Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8229367Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8229545Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8229703Z #35 method_vectorcall from :0 2024-04-03T16:26:27.8229900Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8230057Z #37 method_vectorcall from :0 2024-04-03T16:26:27.8230233Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8230406Z #39 method_vectorcall from :0 2024-04-03T16:26:27.8230585Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8230758Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8230953Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8231109Z #43 method_vectorcall from :0 2024-04-03T16:26:27.8231390Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8231563Z #45 method_vectorcall from :0 2024-04-03T16:26:27.8231738Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8231912Z #47 method_vectorcall from :0 2024-04-03T16:26:27.8232095Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8232249Z #49 method_vectorcall from :0 2024-04-03T16:26:27.8232421Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.8232598Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8232769Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8233046Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8233217Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8233364Z #55 slot_tp_call from :0 2024-04-03T16:26:27.8233589Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8233769Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8233941Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8234137Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8234318Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8234500Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8234690Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8234876Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8235034Z #64 method_vectorcall from :0 2024-04-03T16:26:27.8235234Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8235400Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8235669Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8235840Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8235986Z #69 slot_tp_call from :0 2024-04-03T16:26:27.8236159Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.8236338Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8236507Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8236702Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8236857Z #74 method_vectorcall from :0 2024-04-03T16:26:27.8237034Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8237230Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8237408Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8237577Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8237776Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8237951Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8238131Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8238320Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8238498Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8238667Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8238862Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8239131Z #86 method_vectorcall from :0 2024-04-03T16:26:27.8239419Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8239582Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8239839Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8240027Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8240187Z #91 slot_tp_call from :0 2024-04-03T16:26:27.8240395Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8240593Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8240883Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8241083Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8241276Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8241453Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8241610Z #98 method_vectorcall from :0 2024-04-03T16:26:27.8241807Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8241984Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8242253Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8242446Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8242594Z #103 slot_tp_call from :0 2024-04-03T16:26:27.8242809Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8243009Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8243349Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8243547Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8243720Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8243904Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8244095Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8244274Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8244448Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8244646Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8244809Z #114 method_vectorcall from :0 2024-04-03T16:26:27.8244989Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8245183Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8245450Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8245623Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8245789Z #119 slot_tp_call from :0 2024-04-03T16:26:27.8246001Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8246193Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8246385Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8246567Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8246748Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8246953Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8247109Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.8247290Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8247449Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.8247589Z #129 run_mod from :0 2024-04-03T16:26:27.8247762Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.8248002Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8248222Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8248411Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8248566Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.8248725Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.8248883Z #136 _start from ??:0 2024-04-03T16:26:27.8249013Z " 2024-04-03T16:26:27.8249035Z 2024-04-03T16:26:27.8249370Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.8249990Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.8250020Z 2024-04-03T16:26:27.8250430Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.8251403Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_foreach/test_foreach-273a9009fcdeef67.xml - 2024-04-03T16:26:27.8251682Z =========================== short test summary info ============================ 2024-04-03T16:26:27.8253299Z FAILED [0.0082s] test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 - AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.8254248Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.8254428Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.8255339Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8256422Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8256883Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8257418Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8257839Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8258674Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8259474Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8260345Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8263795Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8264409Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8265233Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8266007Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8267114Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8271711Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8272625Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8273820Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.8278858Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.8279773Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8280626Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.8280793Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.8281036Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8281270Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8281471Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8281641Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8281870Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8282069Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8282235Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8282461Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8282652Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8282832Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8283038Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8283242Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8283423Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8283631Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8283796Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8284026Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8284194Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8284400Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8284586Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8284793Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8284988Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8285211Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8285378Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8285584Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8285767Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8285971Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8286163Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8286369Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8286535Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8286724Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8286929Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8287123Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8287419Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8287617Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8287775Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8288020Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8288227Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8288424Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8288648Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8288840Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8289054Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8289265Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8289472Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8289775Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8289998Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8290191Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8290487Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8290678Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8290834Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8291018Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8291227Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8291425Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8291643Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8291808Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8292112Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8292325Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8292526Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8292723Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8292943Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8293136Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8293356Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8293555Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8293762Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8293970Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8294177Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8294343Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8294562Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8294756Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8295048Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8295258Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8295415Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8295647Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8295869Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8296065Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8296270Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8296479Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8296696Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8296882Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8297097Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8297299Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8297603Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8297807Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8297978Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8298229Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8298445Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8298651Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8298881Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8299081Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8299290Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8299503Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8299716Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8299917Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8300148Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8300321Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8300548Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8300747Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8301042Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8301258Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8301416Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8301748Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8301992Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8302190Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8302402Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8302614Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8302824Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8302989Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.8303184Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.8303351Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.8303498Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.8303680Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.8304040Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8304289Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8304479Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.8304651Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.8304840Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.8304993Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.8305731Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8306557Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8306714Z C++ CapturedTraceback: 2024-04-03T16:26:27.8307425Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8308367Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8308773Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8309241Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8309603Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8310324Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8311065Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8311776Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8314787Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8315473Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8316460Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8320285Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8320858Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8321390Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.8321546Z #18 cfunction_call from :0 2024-04-03T16:26:27.8321775Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8321958Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8322131Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8322428Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8322649Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8322782Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.8323049Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8323181Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8323313Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8323509Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8323680Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8323834Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8324028Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8324200Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8324369Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8324546Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8324705Z #35 method_vectorcall from :0 2024-04-03T16:26:27.8324897Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8325053Z #37 method_vectorcall from :0 2024-04-03T16:26:27.8325230Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8325412Z #39 method_vectorcall from :0 2024-04-03T16:26:27.8325590Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8325759Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8325954Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8326116Z #43 method_vectorcall from :0 2024-04-03T16:26:27.8326294Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8326468Z #45 method_vectorcall from :0 2024-04-03T16:26:27.8326644Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8326799Z #47 method_vectorcall from :0 2024-04-03T16:26:27.8326989Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8327142Z #49 method_vectorcall from :0 2024-04-03T16:26:27.8327296Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.8327490Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8327661Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8327935Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8328108Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8328264Z #55 slot_tp_call from :0 2024-04-03T16:26:27.8328488Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8328668Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8328839Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8329038Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8329208Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8329386Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8329573Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8329749Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8329904Z #64 method_vectorcall from :0 2024-04-03T16:26:27.8330096Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8330269Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8330523Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8330727Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8330864Z #69 slot_tp_call from :0 2024-04-03T16:26:27.8331112Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.8331220Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8331496Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8331671Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8331939Z #74 method_vectorcall from :0 2024-04-03T16:26:27.8332507Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8332912Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8333450Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8333986Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8334498Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8335090Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8335557Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8336186Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8336679Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8337353Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8337735Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8338467Z #86 method_vectorcall from :0 2024-04-03T16:26:27.8338845Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8339368Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8340085Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8340752Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8341093Z #91 slot_tp_call from :0 2024-04-03T16:26:27.8341590Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8342196Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8342854Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8343383Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8343763Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8344275Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8344788Z #98 method_vectorcall from :0 2024-04-03T16:26:27.8345405Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8345786Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8346403Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8347044Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8347530Z #103 slot_tp_call from :0 2024-04-03T16:26:27.8348016Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8348612Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8349143Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8349667Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8350204Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8350714Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8351249Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8351774Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8352297Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8352821Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8353347Z #114 method_vectorcall from :0 2024-04-03T16:26:27.8353818Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8354358Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8354983Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8355600Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8356095Z #119 slot_tp_call from :0 2024-04-03T16:26:27.8356591Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8357166Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8357707Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8358240Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8358766Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8359495Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8359995Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.8360431Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8360893Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.8361333Z #129 run_mod from :0 2024-04-03T16:26:27.8361712Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.8362250Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8362910Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8363468Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8363957Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.8364407Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.8364984Z #136 _start from ??:0 2024-04-03T16:26:27.8365380Z " 2024-04-03T16:26:27.8365552Z 2024-04-03T16:26:27.8365898Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.8367005Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.8367709Z 2024-04-03T16:26:27.8368115Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.8368997Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-04-03T16:26:27.8369860Z ======= 1 failed, 15 passed, 2 skipped, 99 deselected, 2 rerun in 2.03s ======== 2024-04-03T16:26:27.8370564Z Got exit code 1 2024-04-03T16:26:27.8370924Z Retrying... 2024-04-03T16:26:27.8372005Z Test results will be stored in test-reports/python-pytest/test_foreach/test_foreach-e74386081b5ae225.xml 2024-04-03T16:26:27.8373059Z ============================= test session starts ============================== 2024-04-03T16:26:27.8374180Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.4.0 -- /opt/conda/envs/py_3.10/bin/python 2024-04-03T16:26:27.8375070Z cachedir: .pytest_cache 2024-04-03T16:26:27.8376162Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-04-03T16:26:27.8377245Z rootdir: /var/lib/jenkins/workspace 2024-04-03T16:26:27.8377741Z configfile: pytest.ini 2024-04-03T16:26:27.8378697Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-04-03T16:26:27.8379825Z collecting ... collected 2190 items / 116 deselected / 2074 selected 2024-04-03T16:26:27.8380573Z stepcurrent: skipping 116 already run items. 2024-04-03T16:26:27.8381153Z Running 1000 items in this shard 2024-04-03T16:26:27.8381478Z 2024-04-03T16:26:27.8382816Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 [W403 16:26:20.643005276 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8384335Z 2024-04-03T16:26:27.8385119Z [W403 16:26:20.805327188 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8386105Z 2024-04-03T16:26:27.8386882Z [W403 16:26:20.812090187 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8387857Z 2024-04-03T16:26:27.8388650Z [W403 16:26:20.863089460 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8389613Z 2024-04-03T16:26:27.8390400Z [W403 16:26:20.881163858 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8391376Z 2024-04-03T16:26:27.8392154Z [W403 16:26:20.898284582 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8393144Z 2024-04-03T16:26:27.8393918Z [W403 16:26:20.898498465 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8394902Z 2024-04-03T16:26:27.8395678Z [W403 16:26:20.909590037 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8396644Z 2024-04-03T16:26:27.8397434Z [W403 16:26:20.924848524 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8398395Z 2024-04-03T16:26:27.8399353Z [W403 16:26:20.991075930 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8400353Z 2024-04-03T16:26:27.8400635Z ('RERUN', {'yellow': True}) [1.3041s] [ 0%] 2024-04-03T16:26:27.8402518Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 [W403 16:26:20.025073949 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8404050Z 2024-04-03T16:26:27.8404847Z [W403 16:26:20.025376390 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8405816Z 2024-04-03T16:26:27.8406610Z [W403 16:26:20.026006898 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8407581Z 2024-04-03T16:26:27.8408351Z [W403 16:26:20.026223458 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8409451Z 2024-04-03T16:26:27.8410221Z [W403 16:26:20.026531241 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8411254Z 2024-04-03T16:26:27.8412033Z [W403 16:26:20.026714859 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8412995Z 2024-04-03T16:26:27.8413779Z [W403 16:26:20.026876887 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8414738Z 2024-04-03T16:26:27.8415513Z [W403 16:26:20.027039114 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8416495Z 2024-04-03T16:26:27.8417267Z [W403 16:26:20.027556135 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8418245Z 2024-04-03T16:26:27.8419025Z [W403 16:26:20.028907040 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8419990Z 2024-04-03T16:26:27.8420291Z ('RERUN', {'yellow': True}) [0.0086s] [ 0%] 2024-04-03T16:26:27.8422008Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 [W403 16:26:21.035446234 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8423502Z 2024-04-03T16:26:27.8424279Z [W403 16:26:21.035729856 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8425255Z 2024-04-03T16:26:27.8426028Z [W403 16:26:21.036278051 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8427006Z 2024-04-03T16:26:27.8427777Z [W403 16:26:21.036492651 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8428749Z 2024-04-03T16:26:27.8429540Z [W403 16:26:21.036791293 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8430501Z 2024-04-03T16:26:27.8431287Z [W403 16:26:21.036969312 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8432248Z 2024-04-03T16:26:27.8433018Z [W403 16:26:21.037134511 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8434002Z 2024-04-03T16:26:27.8434779Z [W403 16:26:21.037290222 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8435763Z 2024-04-03T16:26:27.8436536Z [W403 16:26:21.038031298 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8437507Z 2024-04-03T16:26:27.8438292Z [W403 16:26:21.039353777 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.8439636Z 2024-04-03T16:26:27.8439805Z FAILED [0.0080s] [ 0%] 2024-04-03T16:26:27.8440101Z 2024-04-03T16:26:27.8440344Z ==================================== RERUNS ==================================== 2024-04-03T16:26:27.8441213Z __ TestForeachCUDA.test_binary_op_list_error_cases__foreach_add_cuda_float16 ___ 2024-04-03T16:26:27.8442017Z Traceback (most recent call last): 2024-04-03T16:26:27.8442951Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 438, in test_binary_op_list_error_cases 2024-04-03T16:26:27.8443917Z foreach_op(tensors1, tensors2) 2024-04-03T16:26:27.8444966Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8446533Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8447820Z C++ CapturedTraceback: 2024-04-03T16:26:27.8448809Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8450615Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8452131Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8453180Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8454186Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8455450Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8456943Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8458462Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8462462Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8466186Z #13 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8467596Z #14 at::native::foreach_tensor_add_list_kernel_slow(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8469223Z #15 at::native::foreach_tensor_add_list_kernel_cuda(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8471134Z #16 at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8476653Z #17 c10::impl::wrap_kernel_functor_unboxed_ > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8482047Z #18 at::_ops::_foreach_add_List::redispatch(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8484126Z #19 torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.8489969Z #20 c10::impl::wrap_kernel_functor_unboxed_ > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.8495500Z #21 at::_ops::_foreach_add_List::call(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8496888Z #22 torch::autograd::THPVariable__foreach_add(_object*, _object*, _object*) from python_torch_functions_1.cpp:0 2024-04-03T16:26:27.8497829Z #23 cfunction_call from :0 2024-04-03T16:26:27.8498329Z #24 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8498902Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8499457Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8499969Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8500428Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8500966Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8501480Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8501939Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8502472Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8502984Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8503443Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8503981Z #35 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8517751Z #36 PyObject_Call from ??:0 2024-04-03T16:26:27.8518359Z #37 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8518904Z #38 method_vectorcall from :0 2024-04-03T16:26:27.8519773Z #39 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8520317Z #40 method_vectorcall from :0 2024-04-03T16:26:27.8520791Z #41 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8521334Z #42 method_vectorcall from :0 2024-04-03T16:26:27.8521817Z #43 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8522335Z #44 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8522865Z #45 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8523393Z #46 method_vectorcall from :0 2024-04-03T16:26:27.8523857Z #47 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8524370Z #48 method_vectorcall from :0 2024-04-03T16:26:27.8524849Z #49 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8525346Z #50 method_vectorcall from :0 2024-04-03T16:26:27.8525819Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8526333Z #52 method_vectorcall from :0 2024-04-03T16:26:27.8526776Z #53 PyObject_Call from ??:0 2024-04-03T16:26:27.8527247Z #54 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8527777Z #55 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8528374Z #56 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8528995Z #57 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8529513Z #58 slot_tp_call from :0 2024-04-03T16:26:27.8529975Z #59 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8530553Z #60 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8531090Z #61 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8531805Z #62 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8532376Z #63 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8532903Z #64 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8533416Z #65 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8533940Z #66 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8534462Z #67 method_vectorcall from :0 2024-04-03T16:26:27.8534928Z #68 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8535458Z #69 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8536062Z #70 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8536668Z #71 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8537160Z #72 slot_tp_call from :0 2024-04-03T16:26:27.8537582Z #73 PyObject_Call from ??:0 2024-04-03T16:26:27.8538181Z #74 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8538717Z #75 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8539239Z #76 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8539747Z #77 method_vectorcall from :0 2024-04-03T16:26:27.8540233Z #78 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8540768Z #79 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8541279Z #80 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8541808Z #81 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8542330Z #82 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8542845Z #83 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8543376Z #84 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8543911Z #85 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8544417Z #86 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8544948Z #87 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8545472Z #88 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8545982Z #89 method_vectorcall from :0 2024-04-03T16:26:27.8546464Z #90 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8546999Z #91 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8547595Z #92 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8548222Z #93 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8548715Z #94 slot_tp_call from :0 2024-04-03T16:26:27.8549180Z #95 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8549755Z #96 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8550284Z #97 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8550792Z #98 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8551319Z #99 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8551852Z #100 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8552365Z #101 method_vectorcall from :0 2024-04-03T16:26:27.8552851Z #102 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8553395Z #103 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8554006Z #104 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8554636Z #105 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8555135Z #106 slot_tp_call from :0 2024-04-03T16:26:27.8555605Z #107 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8556199Z #108 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8556743Z #109 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8557252Z #110 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8557793Z #111 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8558326Z #112 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8558848Z #113 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8559595Z #114 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8560136Z #115 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8560652Z #116 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8561175Z #117 method_vectorcall from :0 2024-04-03T16:26:27.8561663Z #118 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8562186Z #119 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8562823Z #120 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8563458Z #121 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8564041Z #122 slot_tp_call from :0 2024-04-03T16:26:27.8564706Z #123 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8565176Z #124 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8565704Z #125 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8566233Z #126 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8566766Z #127 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8567293Z #128 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8567792Z #129 _PyEval_Vector from :0 2024-04-03T16:26:27.8568246Z #130 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8568714Z #131 run_eval_code_obj from :0 2024-04-03T16:26:27.8569141Z #132 run_mod from :0 2024-04-03T16:26:27.8569547Z #133 pyrun_file.cold from :0 2024-04-03T16:26:27.8570068Z #134 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8570853Z #135 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8571426Z #136 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8571918Z #137 Py_BytesMain from ??:0 2024-04-03T16:26:27.8572355Z #138 __libc_start_main from ??:0 2024-04-03T16:26:27.8572820Z #139 _start from ??:0 2024-04-03T16:26:27.8573078Z 2024-04-03T16:26:27.8573098Z 2024-04-03T16:26:27.8573477Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.8574043Z 2024-04-03T16:26:27.8574788Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8576369Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8577513Z C++ CapturedTraceback: 2024-04-03T16:26:27.8578486Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8580309Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8581831Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8582865Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8583855Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8585132Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8586616Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8588122Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8592102Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8595998Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8597830Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8602841Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8607060Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8608285Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.8609148Z #18 cfunction_call from :0 2024-04-03T16:26:27.8609639Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8610221Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8610737Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8611454Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8611985Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8612476Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.8612947Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8613491Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8613984Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8614455Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8614986Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8615474Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8615944Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8616479Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8616963Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8617432Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8617954Z #35 method_vectorcall from :0 2024-04-03T16:26:27.8618419Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8618936Z #37 method_vectorcall from :0 2024-04-03T16:26:27.8619428Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8619920Z #39 method_vectorcall from :0 2024-04-03T16:26:27.8620403Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8620935Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8621451Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8621965Z #43 method_vectorcall from :0 2024-04-03T16:26:27.8622449Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8622947Z #45 method_vectorcall from :0 2024-04-03T16:26:27.8623428Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8623941Z #47 method_vectorcall from :0 2024-04-03T16:26:27.8624400Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8624909Z #49 method_vectorcall from :0 2024-04-03T16:26:27.8625369Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.8625823Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8626353Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8626954Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8627572Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8628066Z #55 slot_tp_call from :0 2024-04-03T16:26:27.8628542Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8629107Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8629633Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8630162Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8630676Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8631206Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8631736Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8632241Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8632755Z #64 method_vectorcall from :0 2024-04-03T16:26:27.8633231Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8633842Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8634345Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8634964Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8635457Z #69 slot_tp_call from :0 2024-04-03T16:26:27.8635931Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.8636409Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8636919Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8637548Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8638087Z #74 method_vectorcall from :0 2024-04-03T16:26:27.8638549Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8639280Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8639801Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8640311Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8640837Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8641367Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8641877Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8642400Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8642924Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8643434Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8644128Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8644739Z #86 method_vectorcall from :0 2024-04-03T16:26:27.8645124Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8645634Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8646240Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8646849Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8647342Z #91 slot_tp_call from :0 2024-04-03T16:26:27.8647819Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8648378Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8648907Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8649435Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8649953Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8650478Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8650995Z #98 method_vectorcall from :0 2024-04-03T16:26:27.8651457Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8651998Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8652615Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8653236Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8653739Z #103 slot_tp_call from :0 2024-04-03T16:26:27.8654238Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8654812Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8655361Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8655894Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8656421Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8656954Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8657494Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8658027Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8658552Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8659079Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8659601Z #114 method_vectorcall from :0 2024-04-03T16:26:27.8660079Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8660619Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8661234Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8661861Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8662365Z #119 slot_tp_call from :0 2024-04-03T16:26:27.8662846Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8663415Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8663952Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8664477Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8665003Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8665538Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8666052Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.8666485Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8666953Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.8667389Z #129 run_mod from :0 2024-04-03T16:26:27.8667774Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.8668318Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8668959Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8669523Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8670128Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.8670603Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.8671043Z #136 _start from ??:0 2024-04-03T16:26:27.8671312Z 2024-04-03T16:26:27.8671331Z 2024-04-03T16:26:27.8671689Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.8672252Z 2024-04-03T16:26:27.8672443Z Traceback (most recent call last): 2024-04-03T16:26:27.8673372Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 440, in test_binary_op_list_error_cases 2024-04-03T16:26:27.8674468Z with self.assertRaisesRegex(type(e), re.escape(str(e))): 2024-04-03T16:26:27.8675439Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 239, in __exit__ 2024-04-03T16:26:27.8676458Z self._raiseFailure('"{}" does not match "{}"'.format( 2024-04-03T16:26:27.8677540Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 163, in _raiseFailure 2024-04-03T16:26:27.8678473Z raise self.test_case.failureException(msg) 2024-04-03T16:26:27.8679977Z AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.8681839Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.8683103Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.8684206Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8686245Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8687974Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8689149Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8690289Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8691733Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8693434Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8695179Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8699675Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8703895Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8705481Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8707243Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8709344Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8715468Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8721368Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8723652Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.8729957Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.8735710Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8737290Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.8738315Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.8738879Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8739521Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8740111Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8740675Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8741225Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8741825Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8742382Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8742928Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8743516Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8744069Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8744614Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8745194Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8745746Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8746291Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8746847Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8747396Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8747969Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8748495Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8749058Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8749617Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8750197Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8750789Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8751493Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8752052Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8752620Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8753165Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8753712Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8754256Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8754820Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8755303Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8755843Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8756433Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8757084Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8757888Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8758438Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8758951Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8759762Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8760370Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8760948Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8761545Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8762141Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8762735Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8763315Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8763885Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8764432Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8765015Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8765679Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8766363Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8766893Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8767366Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8767911Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8768497Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8769093Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8769660Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8770186Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8770781Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8771376Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8771948Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8772540Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8773136Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8773709Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8774298Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8774904Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8775478Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8776067Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8776646Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8777170Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8777761Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8778428Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8779088Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8779638Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8780167Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8780786Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8781380Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8781972Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8782173Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8782408Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8782601Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8782822Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8783182Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8783496Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8783700Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8783870Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8784105Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8784317Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8784533Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8784746Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8784948Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8785171Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8785488Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8785701Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8785919Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8786136Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8786308Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8786537Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8786737Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8787038Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8787239Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8787402Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8787650Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8787856Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8788054Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8788278Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8788487Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8788697Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8788878Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.8789059Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.8789226Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.8789386Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.8789552Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.8789807Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8790056Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8790244Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.8790425Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.8790600Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.8790754Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.8791522Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8792338Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8792498Z C++ CapturedTraceback: 2024-04-03T16:26:27.8793240Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8794167Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8794579Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8795024Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8795390Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8796137Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8796721Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8797555Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8800832Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8801518Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8802655Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8806180Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8806705Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8807250Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.8807427Z #18 cfunction_call from :0 2024-04-03T16:26:27.8807644Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8807831Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8808024Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8808206Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8808376Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8808549Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.8808732Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8808904Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8809073Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8809249Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8809419Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8809592Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8809770Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8809952Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8810124Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8810302Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8810483Z #35 method_vectorcall from :0 2024-04-03T16:26:27.8810659Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8810811Z #37 method_vectorcall from :0 2024-04-03T16:26:27.8811050Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8811213Z #39 method_vectorcall from :0 2024-04-03T16:26:27.8811388Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8811572Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8811749Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8811905Z #43 method_vectorcall from :0 2024-04-03T16:26:27.8812096Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8812251Z #45 method_vectorcall from :0 2024-04-03T16:26:27.8812427Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8812593Z #47 method_vectorcall from :0 2024-04-03T16:26:27.8812777Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8812932Z #49 method_vectorcall from :0 2024-04-03T16:26:27.8813104Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.8813399Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8813590Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8813864Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8814036Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8814183Z #55 slot_tp_call from :0 2024-04-03T16:26:27.8814411Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8814589Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8814766Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8814939Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8815109Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8815302Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8815470Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8815745Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8815919Z #64 method_vectorcall from :0 2024-04-03T16:26:27.8816095Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8816265Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8816542Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8816714Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8816861Z #69 slot_tp_call from :0 2024-04-03T16:26:27.8817032Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.8817208Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8817375Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8817568Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8817728Z #74 method_vectorcall from :0 2024-04-03T16:26:27.8817903Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8818090Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8818263Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8818448Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8818632Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8818800Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8818992Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8819171Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8819349Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8819534Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8819708Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8819867Z #86 method_vectorcall from :0 2024-04-03T16:26:27.8820047Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8820213Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8820495Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8820665Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8820811Z #91 slot_tp_call from :0 2024-04-03T16:26:27.8821035Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8821213Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8821389Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8821583Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8821752Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8821926Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8822103Z #98 method_vectorcall from :0 2024-04-03T16:26:27.8822280Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8822457Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8822737Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8822913Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8823063Z #103 slot_tp_call from :0 2024-04-03T16:26:27.8823294Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8823475Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8823665Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8823846Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8824018Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8824221Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8824395Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8824573Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8824759Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8825035Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8825210Z #114 method_vectorcall from :0 2024-04-03T16:26:27.8825403Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8825575Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8825843Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8826029Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8826176Z #119 slot_tp_call from :0 2024-04-03T16:26:27.8826393Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8826591Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8826763Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8826940Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8827247Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8827423Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8827589Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.8827750Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8827906Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.8828061Z #129 run_mod from :0 2024-04-03T16:26:27.8828219Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.8828457Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8828687Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8828857Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8829008Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.8829183Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.8829324Z #136 _start from ??:0 2024-04-03T16:26:27.8829446Z " 2024-04-03T16:26:27.8829468Z 2024-04-03T16:26:27.8829820Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.8830422Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.8830451Z 2024-04-03T16:26:27.8830880Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.8831308Z __ TestForeachCUDA.test_binary_op_list_error_cases__foreach_add_cuda_float16 ___ 2024-04-03T16:26:27.8831484Z Traceback (most recent call last): 2024-04-03T16:26:27.8832101Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 438, in test_binary_op_list_error_cases 2024-04-03T16:26:27.8832268Z foreach_op(tensors1, tensors2) 2024-04-03T16:26:27.8832969Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8833780Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8833936Z C++ CapturedTraceback: 2024-04-03T16:26:27.8834651Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8835608Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8836011Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8836470Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8836832Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8837552Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8838138Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8838882Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8842280Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8842818Z #13 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8843548Z #14 at::native::foreach_tensor_add_list_kernel_slow(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8844256Z #15 at::native::foreach_tensor_add_list_kernel_cuda(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8845388Z #16 at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8849654Z #17 c10::impl::wrap_kernel_functor_unboxed_ > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8850456Z #18 at::_ops::_foreach_add_List::redispatch(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8851568Z #19 torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.8856152Z #20 c10::impl::wrap_kernel_functor_unboxed_ > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.8856803Z #21 at::_ops::_foreach_add_List::call(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8857392Z #22 torch::autograd::THPVariable__foreach_add(_object*, _object*, _object*) from python_torch_functions_1.cpp:0 2024-04-03T16:26:27.8857565Z #23 cfunction_call from :0 2024-04-03T16:26:27.8857775Z #24 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8857959Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8858142Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8858298Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8858492Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8858663Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8858815Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8859014Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8859182Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8859335Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8859525Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8859785Z #35 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8859955Z #36 PyObject_Call from ??:0 2024-04-03T16:26:27.8860150Z #37 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8860307Z #38 method_vectorcall from :0 2024-04-03T16:26:27.8860484Z #39 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8860650Z #40 method_vectorcall from :0 2024-04-03T16:26:27.8860825Z #41 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8860977Z #42 method_vectorcall from :0 2024-04-03T16:26:27.8861166Z #43 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8861335Z #44 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8861510Z #45 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8861680Z #46 method_vectorcall from :0 2024-04-03T16:26:27.8862017Z #47 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8862172Z #48 method_vectorcall from :0 2024-04-03T16:26:27.8862361Z #49 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8862512Z #50 method_vectorcall from :0 2024-04-03T16:26:27.8862703Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8862859Z #52 method_vectorcall from :0 2024-04-03T16:26:27.8863011Z #53 PyObject_Call from ??:0 2024-04-03T16:26:27.8863201Z #54 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8863371Z #55 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8863629Z #56 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8863812Z #57 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8863959Z #58 slot_tp_call from :0 2024-04-03T16:26:27.8864168Z #59 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8864362Z #60 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8864531Z #61 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8864706Z #62 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8864896Z #63 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8865070Z #64 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8865240Z #65 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8865440Z #66 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8865595Z #67 method_vectorcall from :0 2024-04-03T16:26:27.8865768Z #68 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8865952Z #69 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8866207Z #70 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8866391Z #71 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8866535Z #72 slot_tp_call from :0 2024-04-03T16:26:27.8866687Z #73 PyObject_Call from ??:0 2024-04-03T16:26:27.8866877Z #74 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8867046Z #75 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8867220Z #76 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8867391Z #77 method_vectorcall from :0 2024-04-03T16:26:27.8867577Z #78 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8867747Z #79 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8867941Z #80 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8868111Z #81 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8868292Z #82 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8868474Z #83 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8868646Z #84 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8868811Z #85 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8869004Z #86 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8869170Z #87 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8869344Z #88 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8869517Z #89 method_vectorcall from :0 2024-04-03T16:26:27.8869693Z #90 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8869875Z #91 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8870128Z #92 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8870295Z #93 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8870464Z #94 slot_tp_call from :0 2024-04-03T16:26:27.8870668Z #95 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8870844Z #96 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8871110Z #97 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8871302Z #98 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8871464Z #99 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8871661Z #100 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8871817Z #101 method_vectorcall from :0 2024-04-03T16:26:27.8871990Z #102 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8872176Z #103 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8872441Z #104 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8872614Z #105 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8872779Z #106 slot_tp_call from :0 2024-04-03T16:26:27.8872991Z #107 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8873168Z #108 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8873458Z #109 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8873637Z #110 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8873827Z #111 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8874010Z #112 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8874182Z #113 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8874375Z #114 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8874550Z #115 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8874729Z #116 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8874902Z #117 method_vectorcall from :0 2024-04-03T16:26:27.8875079Z #118 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8875252Z #119 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8875528Z #120 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8875700Z #121 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8875847Z #122 slot_tp_call from :0 2024-04-03T16:26:27.8876078Z #123 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8876265Z #124 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8876437Z #125 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8876630Z #126 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8876810Z #127 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8876989Z #128 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8877154Z #129 _PyEval_Vector from :0 2024-04-03T16:26:27.8877315Z #130 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8877484Z #131 run_eval_code_obj from :0 2024-04-03T16:26:27.8877623Z #132 run_mod from :0 2024-04-03T16:26:27.8877775Z #133 pyrun_file.cold from :0 2024-04-03T16:26:27.8878028Z #134 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8878247Z #135 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8878417Z #136 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8878585Z #137 Py_BytesMain from ??:0 2024-04-03T16:26:27.8878742Z #138 __libc_start_main from ??:0 2024-04-03T16:26:27.8878882Z #139 _start from ??:0 2024-04-03T16:26:27.8878912Z 2024-04-03T16:26:27.8878932Z 2024-04-03T16:26:27.8879523Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.8879547Z 2024-04-03T16:26:27.8880258Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8881076Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8881233Z C++ CapturedTraceback: 2024-04-03T16:26:27.8881954Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8882895Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8883289Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8883730Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8884118Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8884981Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8885599Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8886340Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8889388Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8890208Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8891176Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8894707Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8895221Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8895774Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.8895932Z #18 cfunction_call from :0 2024-04-03T16:26:27.8896159Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8896343Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8896518Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8896713Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8896884Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8897039Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.8897235Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8897402Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8897566Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8897758Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8897928Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8898088Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8898283Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8898451Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8898600Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8898789Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8898946Z #35 method_vectorcall from :0 2024-04-03T16:26:27.8899122Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8899295Z #37 method_vectorcall from :0 2024-04-03T16:26:27.8899472Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8899626Z #39 method_vectorcall from :0 2024-04-03T16:26:27.8899816Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8899987Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8900176Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8900340Z #43 method_vectorcall from :0 2024-04-03T16:26:27.8900515Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8900687Z #45 method_vectorcall from :0 2024-04-03T16:26:27.8900953Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8901124Z #47 method_vectorcall from :0 2024-04-03T16:26:27.8901314Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8901471Z #49 method_vectorcall from :0 2024-04-03T16:26:27.8901622Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.8901813Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8901983Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8902241Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8902432Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8902578Z #55 slot_tp_call from :0 2024-04-03T16:26:27.8902784Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8902981Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8903149Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8903423Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8903602Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8903780Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8903970Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8904147Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8904303Z #64 method_vectorcall from :0 2024-04-03T16:26:27.8904491Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8904660Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8904916Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8905103Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8905251Z #69 slot_tp_call from :0 2024-04-03T16:26:27.8905405Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.8905597Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8905766Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8905941Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8906123Z #74 method_vectorcall from :0 2024-04-03T16:26:27.8906299Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8906466Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8906661Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8906838Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8907015Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8907202Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8907380Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8907549Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8907743Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8907916Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8908110Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8908268Z #86 method_vectorcall from :0 2024-04-03T16:26:27.8908446Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8908631Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8908886Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8909064Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8909226Z #91 slot_tp_call from :0 2024-04-03T16:26:27.8909429Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8909612Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8909801Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8909976Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8910145Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8910335Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8910489Z #98 method_vectorcall from :0 2024-04-03T16:26:27.8910664Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8910858Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8911177Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8911358Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8911526Z #103 slot_tp_call from :0 2024-04-03T16:26:27.8911741Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8911948Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8912121Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8912297Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8912581Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8912776Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8912949Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8913145Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8913317Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8913493Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8913665Z #114 method_vectorcall from :0 2024-04-03T16:26:27.8913842Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8914017Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8914296Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8914468Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8914614Z #119 slot_tp_call from :0 2024-04-03T16:26:27.8914948Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8915129Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8915303Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8915506Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8915681Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8915875Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8916026Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.8916188Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.8916357Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.8916495Z #129 run_mod from :0 2024-04-03T16:26:27.8916651Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.8916904Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.8917120Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.8917292Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.8917467Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.8917628Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.8917779Z #136 _start from ??:0 2024-04-03T16:26:27.8917799Z 2024-04-03T16:26:27.8917818Z 2024-04-03T16:26:27.8918200Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.8918221Z 2024-04-03T16:26:27.8918396Z Traceback (most recent call last): 2024-04-03T16:26:27.8919139Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 440, in test_binary_op_list_error_cases 2024-04-03T16:26:27.8919510Z with self.assertRaisesRegex(type(e), re.escape(str(e))): 2024-04-03T16:26:27.8919993Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 239, in __exit__ 2024-04-03T16:26:27.8920356Z self._raiseFailure('"{}" does not match "{}"'.format( 2024-04-03T16:26:27.8920872Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 163, in _raiseFailure 2024-04-03T16:26:27.8921097Z raise self.test_case.failureException(msg) 2024-04-03T16:26:27.8922006Z AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.8922936Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.8923120Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.8923917Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8924986Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8925462Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8925977Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8926393Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8927246Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8928057Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8928975Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8932401Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8933165Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8933959Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8934751Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8935863Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8940533Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.8941434Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8942628Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.8947568Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.8948298Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.8949068Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.8949252Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.8949500Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8949716Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8949913Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8950099Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8950309Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8950521Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8950685Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8950892Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8951103Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8951365Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8951574Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8951783Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8951951Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8952159Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8952336Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8952540Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8952703Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8952923Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8953088Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8953286Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8953497Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8953700Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8953879Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8954084Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8954259Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8954477Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8954689Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8954894Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8955080Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8955243Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8955450Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8955660Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8955936Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8956131Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8956303Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8956532Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8956737Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8956949Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8957167Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8957373Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8957575Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8957773Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8957990Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8958155Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8958356Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8958563Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8958840Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8959140Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8959393Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8959559Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.8959767Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8959975Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8960190Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8960356Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8960572Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8960882Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8961121Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8961311Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8961512Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8961716Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8961918Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8962110Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8962329Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8962519Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8962722Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8962905Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8963230Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8963423Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8963717Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8963915Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8964074Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8964322Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8964524Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8964731Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8964934Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8965124Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8965358Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8965531Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8965746Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8965961Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8966254Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8966453Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8966631Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8966869Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8967079Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8967293Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8967503Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8967702Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8967925Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8968121Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8968339Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8968537Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8968743Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8968928Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.8969148Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8969346Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8969650Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.8969853Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.8970008Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.8970256Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.8970467Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8970661Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8970886Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8971080Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.8971287Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.8971468Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.8971645Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.8971826Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.8971983Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.8972145Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.8972416Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8972734Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.8972938Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.8973119Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.8973290Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.8973438Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.8974196Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.8974994Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.8975153Z C++ CapturedTraceback: 2024-04-03T16:26:27.8975885Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.8976964Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.8977384Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.8977826Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.8978187Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.8978918Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.8979487Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8980249Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8983315Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.8984006Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8984973Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8988494Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.8989019Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.8989554Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.8989726Z #18 cfunction_call from :0 2024-04-03T16:26:27.8989939Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8990125Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8990326Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8990507Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8990678Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8990948Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.8991142Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8991327Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8991484Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.8991660Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8991845Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8991998Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.8992173Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8992355Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8992507Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.8992683Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8992855Z #35 method_vectorcall from :0 2024-04-03T16:26:27.8993029Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8993285Z #37 method_vectorcall from :0 2024-04-03T16:26:27.8993477Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8993630Z #39 method_vectorcall from :0 2024-04-03T16:26:27.8993811Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8993997Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8994170Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8994323Z #43 method_vectorcall from :0 2024-04-03T16:26:27.8994516Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8994670Z #45 method_vectorcall from :0 2024-04-03T16:26:27.8994843Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8995014Z #47 method_vectorcall from :0 2024-04-03T16:26:27.8995188Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8995356Z #49 method_vectorcall from :0 2024-04-03T16:26:27.8995508Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.8995681Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8995863Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8996128Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8996297Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8996458Z #55 slot_tp_call from :0 2024-04-03T16:26:27.8996670Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.8996848Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8997028Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8997203Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8997371Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8997560Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8997727Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8997902Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8998076Z #64 method_vectorcall from :0 2024-04-03T16:26:27.8998252Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8998417Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.8998686Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.8998867Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.8999144Z #69 slot_tp_call from :0 2024-04-03T16:26:27.8999460Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.8999638Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.8999831Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9000006Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9000159Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9000349Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9000517Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9000690Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9000872Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9001047Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9001216Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9001407Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9001579Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9001755Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9001950Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9002127Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9002280Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9002602Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9002793Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9003072Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9003241Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9003386Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9003606Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9003779Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9003945Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9004137Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9004305Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9004483Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9004655Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9004953Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9005131Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9005411Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9005590Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9005737Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9005965Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9006145Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9006318Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9006507Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9006676Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9006867Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9007039Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9007220Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9007403Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9007581Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9007750Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9007942Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9008112Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9008386Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9008572Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9008721Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9008933Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9009129Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9009303Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9009478Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9009664Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9009842Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9009994Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9010168Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9010316Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9010482Z #129 run_mod from :0 2024-04-03T16:26:27.9010638Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9010872Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9011146Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9011330Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9011485Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9011653Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9011793Z #136 _start from ??:0 2024-04-03T16:26:27.9011916Z " 2024-04-03T16:26:27.9011938Z 2024-04-03T16:26:27.9012291Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.9012898Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.9012919Z 2024-04-03T16:26:27.9013328Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.9013587Z =================================== FAILURES =================================== 2024-04-03T16:26:27.9014009Z __ TestForeachCUDA.test_binary_op_list_error_cases__foreach_add_cuda_float16 ___ 2024-04-03T16:26:27.9014208Z Traceback (most recent call last): 2024-04-03T16:26:27.9014904Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 438, in test_binary_op_list_error_cases 2024-04-03T16:26:27.9015088Z foreach_op(tensors1, tensors2) 2024-04-03T16:26:27.9015813Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9016609Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9016762Z C++ CapturedTraceback: 2024-04-03T16:26:27.9017488Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9018416Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9018940Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9019381Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9019747Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9020478Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9021048Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9021790Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9024860Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9025377Z #13 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9026107Z #14 at::native::foreach_tensor_add_list_kernel_slow(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9026817Z #15 at::native::foreach_tensor_add_list_kernel_cuda(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9027843Z #16 at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9032097Z #17 c10::impl::wrap_kernel_functor_unboxed_ > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9032910Z #18 at::_ops::_foreach_add_List::redispatch(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9034000Z #19 torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.9038684Z #20 c10::impl::wrap_kernel_functor_unboxed_ > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.9039600Z #21 at::_ops::_foreach_add_List::call(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9040221Z #22 torch::autograd::THPVariable__foreach_add(_object*, _object*, _object*) from python_torch_functions_1.cpp:0 2024-04-03T16:26:27.9040377Z #23 cfunction_call from :0 2024-04-03T16:26:27.9040588Z #24 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9040790Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9040960Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9041117Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9041319Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9041489Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9041656Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9041836Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9042005Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9042175Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9042363Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9042532Z #35 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9042701Z #36 PyObject_Call from ??:0 2024-04-03T16:26:27.9042880Z #37 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9043041Z #38 method_vectorcall from :0 2024-04-03T16:26:27.9043232Z #39 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9043387Z #40 method_vectorcall from :0 2024-04-03T16:26:27.9043559Z #41 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9043730Z #42 method_vectorcall from :0 2024-04-03T16:26:27.9043905Z #43 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9044075Z #44 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9044263Z #45 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9044419Z #46 method_vectorcall from :0 2024-04-03T16:26:27.9044593Z #47 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9044770Z #48 method_vectorcall from :0 2024-04-03T16:26:27.9044947Z #49 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9045128Z #50 method_vectorcall from :0 2024-04-03T16:26:27.9045304Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9045456Z #52 method_vectorcall from :0 2024-04-03T16:26:27.9045621Z #53 PyObject_Call from ??:0 2024-04-03T16:26:27.9045807Z #54 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9045979Z #55 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9046252Z #56 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9046424Z #57 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9046573Z #58 slot_tp_call from :0 2024-04-03T16:26:27.9046795Z #59 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9046973Z #60 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9047141Z #61 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9047335Z #62 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9047502Z #63 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9047677Z #64 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9047859Z #65 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9048045Z #66 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9048199Z #67 method_vectorcall from :0 2024-04-03T16:26:27.9048391Z #68 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9048691Z #69 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9048975Z #70 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9049159Z #71 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9049305Z #72 slot_tp_call from :0 2024-04-03T16:26:27.9049475Z #73 PyObject_Call from ??:0 2024-04-03T16:26:27.9049653Z #74 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9049821Z #75 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9050010Z #76 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9050167Z #77 method_vectorcall from :0 2024-04-03T16:26:27.9050341Z #78 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9050520Z #79 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9050694Z #80 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9050864Z #81 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9051183Z #82 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9051355Z #83 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9051533Z #84 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9051718Z #85 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9051894Z #86 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9052063Z #87 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9052254Z #88 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9052412Z #89 method_vectorcall from :0 2024-04-03T16:26:27.9052583Z #90 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9052763Z #91 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9053016Z #92 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9053201Z #93 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9053347Z #94 slot_tp_call from :0 2024-04-03T16:26:27.9053553Z #95 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9053746Z #96 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9053928Z #97 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9054105Z #98 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9054287Z #99 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9054475Z #100 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9054636Z #101 method_vectorcall from :0 2024-04-03T16:26:27.9054831Z #102 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9055007Z #103 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9055269Z #104 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9055462Z #105 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9055607Z #106 slot_tp_call from :0 2024-04-03T16:26:27.9055820Z #107 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9056016Z #108 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9056188Z #109 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9056362Z #110 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9056547Z #111 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9056729Z #112 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9056910Z #113 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9057087Z #114 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9057269Z #115 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9057460Z #116 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9057619Z #117 method_vectorcall from :0 2024-04-03T16:26:27.9057797Z #118 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9057982Z #119 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9058246Z #120 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9058419Z #121 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9058583Z #122 slot_tp_call from :0 2024-04-03T16:26:27.9058795Z #123 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9058972Z #124 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9059164Z #125 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9059341Z #126 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9059522Z #127 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9059715Z #128 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9059866Z #129 _PyEval_Vector from :0 2024-04-03T16:26:27.9060028Z #130 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9060293Z #131 run_eval_code_obj from :0 2024-04-03T16:26:27.9060448Z #132 run_mod from :0 2024-04-03T16:26:27.9060623Z #133 pyrun_file.cold from :0 2024-04-03T16:26:27.9060865Z #134 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9061080Z #135 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9061269Z #136 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9061427Z #137 Py_BytesMain from ??:0 2024-04-03T16:26:27.9061588Z #138 __libc_start_main from ??:0 2024-04-03T16:26:27.9061743Z #139 _start from ??:0 2024-04-03T16:26:27.9061764Z 2024-04-03T16:26:27.9061784Z 2024-04-03T16:26:27.9062151Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.9062171Z 2024-04-03T16:26:27.9062995Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9063822Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9063983Z C++ CapturedTraceback: 2024-04-03T16:26:27.9064712Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9065640Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9066037Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9066494Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9066855Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9067601Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9068175Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9068920Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9071984Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9072666Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9073651Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9077167Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9077692Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9078222Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9078472Z #18 cfunction_call from :0 2024-04-03T16:26:27.9078708Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9078892Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9079269Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9079452Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9079619Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9079796Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9079973Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9080143Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9080312Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9080492Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9080661Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9080984Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9081161Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9081327Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9081497Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9081679Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9081839Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9082023Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9082177Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9082351Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9082524Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9082699Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9082873Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9083064Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9083218Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9083409Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9083563Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9083747Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9083916Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9084090Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9084244Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9084420Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9084600Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9084771Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9085045Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9085216Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9085361Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9085584Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9085762Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9085931Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9086118Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9086286Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9086459Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9086655Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9086827Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9086994Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9087178Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9087345Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9087613Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9087781Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9087926Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9088090Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9088263Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9088429Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9088619Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9088775Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9088948Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9089133Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9089317Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9089486Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9089678Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9089845Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9090179Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9090385Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9090558Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9090746Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9090924Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9091082Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9091274Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9091445Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9091700Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9091886Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9092031Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9092235Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9092567Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9092737Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9092912Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9093103Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9093279Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9093436Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9093627Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9093806Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9094071Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9094263Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9094411Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9094621Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9094816Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9094991Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9095187Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9095369Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9095546Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9095732Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9095919Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9096091Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9096285Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9096448Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9096624Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9096813Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9097080Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9097254Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9097417Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9097628Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9097804Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9097993Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9098183Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9098371Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9098552Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9098708Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9098889Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9099042Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9099179Z #129 run_mod from :0 2024-04-03T16:26:27.9099347Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9099585Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9099804Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9099983Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9100135Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9100292Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9100447Z #136 _start from ??:0 2024-04-03T16:26:27.9100468Z 2024-04-03T16:26:27.9100488Z 2024-04-03T16:26:27.9100851Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.9100879Z 2024-04-03T16:26:27.9101067Z Traceback (most recent call last): 2024-04-03T16:26:27.9101675Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 440, in test_binary_op_list_error_cases 2024-04-03T16:26:27.9102119Z with self.assertRaisesRegex(type(e), re.escape(str(e))): 2024-04-03T16:26:27.9102635Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 239, in __exit__ 2024-04-03T16:26:27.9102978Z self._raiseFailure('"{}" does not match "{}"'.format( 2024-04-03T16:26:27.9103495Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 163, in _raiseFailure 2024-04-03T16:26:27.9103734Z raise self.test_case.failureException(msg) 2024-04-03T16:26:27.9104642Z AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.9105564Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.9105846Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.9106658Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9107741Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9108195Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9108711Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9109137Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9109963Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9110663Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9111592Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9115046Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9115655Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9116464Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9117260Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9118379Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9123267Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9124200Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9125409Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9130334Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9131181Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9131858Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.9132047Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.9132289Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9132505Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9132718Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9132889Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9133114Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9133309Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9133473Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9133692Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9133885Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9134048Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9134267Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9134458Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9134636Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9134863Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9135031Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9135240Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9135423Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9135628Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9135790Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9136007Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9136204Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9136409Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9136595Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9136799Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9136978Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9137179Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9137355Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9137577Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9137740Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9137897Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9138206Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9138414Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9138694Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9138911Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9139067Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9139294Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9139556Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9139718Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9139919Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9140128Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9140331Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9140657Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9140862Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9141028Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9141253Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9141441Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9141714Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9141919Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9142075Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9142242Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9142461Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9142652Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9142854Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9143036Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9143240Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9143437Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9143653Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9143842Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9144060Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9144249Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9144451Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9144658Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9144863Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9145054Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9145266Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9145431Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9145638Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9145848Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9146123Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9146324Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9146495Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9146721Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9146930Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9147138Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9147339Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9147545Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9147765Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9147931Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9148157Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9148360Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9148643Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9148861Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9149032Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9149267Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9149491Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9149778Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9150004Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9150287Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9150523Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9150634Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9150924Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9151043Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9151267Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9151441Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9151647Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9151860Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9152244Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9152443Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9152616Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9152852Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9153061Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9153276Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9153482Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9153679Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9153906Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9154070Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.9154262Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.9154429Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.9154576Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.9154756Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.9155010Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9155247Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9155447Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.9155608Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.9155786Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.9155950Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.9156691Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9157493Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9157668Z C++ CapturedTraceback: 2024-04-03T16:26:27.9158384Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9159516Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9159934Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9160384Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9160760Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9161483Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9162052Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9162809Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9165991Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9166717Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9167684Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9171224Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9171858Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9172408Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9172563Z #18 cfunction_call from :0 2024-04-03T16:26:27.9172771Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9172973Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9173146Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9173324Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9173509Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9173679Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9173856Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9174039Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9174202Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9174398Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9174568Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9174721Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9174912Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9175081Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9175233Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9175424Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9175582Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9175760Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9175931Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9176104Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9176258Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9176458Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9176630Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9176806Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9176980Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9177155Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9177309Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9177501Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9177655Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9177831Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9178003Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9178155Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9178343Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9178512Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9178769Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9178952Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9179095Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9179312Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9179508Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9179676Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9179941Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9180144Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9180318Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9180487Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9180677Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9180830Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9181006Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9181193Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9181449Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9181617Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9181779Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9181931Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9182223Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9182392Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9182568Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9182739Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9182920Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9183088Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9183278Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9183444Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9183619Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9183804Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9183980Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9184151Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9184346Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9184514Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9184688Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9184860Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9185043Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9185211Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9185488Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9185663Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9185824Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9186032Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9186211Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9186392Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9186567Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9186734Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9186925Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9187077Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9187251Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9187443Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9187708Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9187886Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9188050Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9188263Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9188450Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9188640Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9188819Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9188991Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9189184Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9189356Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9189549Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9189721Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9189900Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9190070Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9190247Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9190421Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9190712Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9190886Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9191032Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9191359Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9191554Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9191723Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9191914Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9192083Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9192258Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9192422Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9192583Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9192736Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9192890Z #129 run_mod from :0 2024-04-03T16:26:27.9193044Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9193295Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9193512Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9193785Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9193952Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9194108Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9194255Z #136 _start from ??:0 2024-04-03T16:26:27.9194392Z " 2024-04-03T16:26:27.9194412Z 2024-04-03T16:26:27.9194745Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.9195340Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.9195361Z 2024-04-03T16:26:27.9195783Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.9196724Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_foreach/test_foreach-e74386081b5ae225.xml - 2024-04-03T16:26:27.9197018Z =========================== short test summary info ============================ 2024-04-03T16:26:27.9198633Z FAILED [0.0080s] test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 - AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.9199754Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.9199948Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.9200746Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9201825Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9202277Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9202791Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9203228Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9204069Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9204743Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9205620Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9209217Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9209882Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9210672Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9211508Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9212624Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9217353Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9218240Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9219460Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9224386Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9225125Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9225808Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.9225984Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.9226215Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9226426Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9226634Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9226800Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9227010Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9227215Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9227377Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9227592Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9227796Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9227960Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9228248Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9228469Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9228629Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9228845Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9229008Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9229210Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9229389Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9229593Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9229756Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9229973Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9230159Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9230362Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9230643Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9230850Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9231014Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9231236Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9231400Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9231604Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9231780Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9231942Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9232149Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9232338Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9232614Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9232816Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9232967Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9233195Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9233417Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9233612Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9233817Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9234021Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9234220Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9234410Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9234626Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9234793Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9234996Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9235196Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9235475Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9235674Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9235828Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9236053Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9236291Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9236484Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9236688Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9236881Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9237087Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9237280Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9237504Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9237699Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9237904Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9238114Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9238319Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9238511Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9238736Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9238930Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9239340Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9239510Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9239708Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9240046Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9240351Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9240547Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9240728Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9240959Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9241166Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9241379Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9241589Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9241783Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9242020Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9242191Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9242548Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9242770Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9243054Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9243278Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9243442Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9243677Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9243906Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9244105Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9244320Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9244539Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9244749Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9244948Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9255641Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9255954Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9256203Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9256396Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9256630Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9256858Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9257183Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9257407Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9257572Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9257845Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9258080Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9258300Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9258547Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9258765Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9259000Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9259189Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.9259392Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.9259584Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.9259743Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.9259919Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.9260212Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9260461Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9260662Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.9260854Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.9261042Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.9261198Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.9262016Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9262870Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9263032Z C++ CapturedTraceback: 2024-04-03T16:26:27.9263815Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9264950Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9265418Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9265888Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9266280Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9267066Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9267680Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9268484Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9271915Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9272650Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9273684Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9277500Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9278054Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9278624Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9278799Z #18 cfunction_call from :0 2024-04-03T16:26:27.9279142Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9279577Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9279778Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9279974Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9280150Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9280325Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9280513Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9280709Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9280868Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9281059Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9281253Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9281413Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9281603Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9281798Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9281960Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9282153Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9282346Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9282535Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9282697Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9283055Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9283244Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9283429Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9283616Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9283803Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9283963Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9284156Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9284314Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9284499Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9284672Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9284858Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9285034Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9285191Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9285503Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9285690Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9285961Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9286149Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9286314Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9286531Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9286720Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9286913Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9287101Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9287284Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9287485Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9287665Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9287847Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9288019Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9288208Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9288394Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9288684Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9288868Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9289040Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9289209Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9289394Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9289585Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9289771Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9289928Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9290125Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9290302Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9290491Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9290685Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9290870Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9291049Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9291255Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9291442Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9291631Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9291821Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9292006Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9292166Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9292372Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9292551Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9292820Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9293016Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9293172Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9293405Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9293596Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9293775Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9293972Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9294145Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9294336Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9294507Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9294698Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9294884Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9295272Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9295473Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9295624Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9295869Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9296056Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9296241Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9296442Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9296626Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9296833Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9297016Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9297211Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9297508Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9297700Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9297864Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9298067Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9298261Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9298539Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9298735Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9298886Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9299111Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9299322Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9299507Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9299695Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9299888Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9300077Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9300232Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9300417Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9300581Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9300723Z #129 run_mod from :0 2024-04-03T16:26:27.9300897Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9301150Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9301395Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9301662Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9301834Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9301917Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9302062Z #136 _start from ??:0 2024-04-03T16:26:27.9302185Z " 2024-04-03T16:26:27.9302207Z 2024-04-03T16:26:27.9302572Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.9303213Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.9303235Z 2024-04-03T16:26:27.9303668Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.9303980Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-04-03T16:26:27.9304332Z ================== 1 failed, 116 deselected, 2 rerun in 1.51s ================== 2024-04-03T16:26:27.9304487Z Got exit code 1 2024-04-03T16:26:27.9304630Z Retrying... 2024-04-03T16:26:27.9305393Z Test results will be stored in test-reports/python-pytest/test_foreach/test_foreach-26a998c1bb36d87a.xml 2024-04-03T16:26:27.9305688Z ============================= test session starts ============================== 2024-04-03T16:26:27.9306388Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.4.0 -- /opt/conda/envs/py_3.10/bin/python 2024-04-03T16:26:27.9306547Z cachedir: .pytest_cache 2024-04-03T16:26:27.9307453Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-04-03T16:26:27.9307631Z rootdir: /var/lib/jenkins/workspace 2024-04-03T16:26:27.9307783Z configfile: pytest.ini 2024-04-03T16:26:27.9308564Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-04-03T16:26:27.9308936Z collecting ... collected 2190 items / 116 deselected / 2074 selected 2024-04-03T16:26:27.9309176Z stepcurrent: skipping 116 already run items. 2024-04-03T16:26:27.9309464Z Running 1000 items in this shard 2024-04-03T16:26:27.9309488Z 2024-04-03T16:26:27.9310923Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 [W403 16:26:26.400350752 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9310946Z 2024-04-03T16:26:27.9311865Z [W403 16:26:26.559742460 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9311887Z 2024-04-03T16:26:27.9312764Z [W403 16:26:26.566411631 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9312943Z 2024-04-03T16:26:27.9313713Z [W403 16:26:26.616624963 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9313734Z 2024-04-03T16:26:27.9314581Z [W403 16:26:26.634610250 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9314601Z 2024-04-03T16:26:27.9315456Z [W403 16:26:26.651240448 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9315476Z 2024-04-03T16:26:27.9316308Z [W403 16:26:26.651448777 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9316328Z 2024-04-03T16:26:27.9317169Z [W403 16:26:26.662467989 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9317202Z 2024-04-03T16:26:27.9318053Z [W403 16:26:26.677580372 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9318073Z 2024-04-03T16:26:27.9318915Z [W403 16:26:26.743194109 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9318935Z 2024-04-03T16:26:27.9319450Z ('RERUN', {'yellow': True}) [1.3034s] [ 0%] 2024-04-03T16:26:27.9320865Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 [W403 16:26:26.777088623 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9320887Z 2024-04-03T16:26:27.9321741Z [W403 16:26:26.777386572 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9321761Z 2024-04-03T16:26:27.9322595Z [W403 16:26:26.777990644 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9322624Z 2024-04-03T16:26:27.9323468Z [W403 16:26:26.778203045 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9323499Z 2024-04-03T16:26:27.9324334Z [W403 16:26:26.778497091 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9324355Z 2024-04-03T16:26:27.9325189Z [W403 16:26:26.778670116 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9325209Z 2024-04-03T16:26:27.9326054Z [W403 16:26:26.778826167 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9326075Z 2024-04-03T16:26:27.9326910Z [W403 16:26:26.778989127 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9326938Z 2024-04-03T16:26:27.9327789Z [W403 16:26:26.779476360 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9327954Z 2024-04-03T16:26:27.9328818Z [W403 16:26:26.780826642 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9328839Z 2024-04-03T16:26:27.9329148Z ('RERUN', {'yellow': True}) [0.0083s] [ 0%] 2024-04-03T16:26:27.9330575Z test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 [W403 16:26:26.787060859 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9330596Z 2024-04-03T16:26:27.9331430Z [W403 16:26:26.787334411 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9331603Z 2024-04-03T16:26:27.9332470Z [W403 16:26:26.787825503 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9332491Z 2024-04-03T16:26:27.9333338Z [W403 16:26:26.788055674 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9333358Z 2024-04-03T16:26:27.9334195Z [W403 16:26:26.788569228 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9334215Z 2024-04-03T16:26:27.9335047Z [W403 16:26:26.788745505 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9335067Z 2024-04-03T16:26:27.9335912Z [W403 16:26:26.788902620 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9335933Z 2024-04-03T16:26:27.9336779Z [W403 16:26:26.789055592 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9336800Z 2024-04-03T16:26:27.9337656Z [W403 16:26:26.789730561 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9337677Z 2024-04-03T16:26:27.9338503Z [W403 16:26:26.791043768 Module.cpp:168] symbolizing C++ stack trace for exception; if this hangs, rerun with TORCH_DISABLE_ADDR2LINE=1... 2024-04-03T16:26:27.9338523Z 2024-04-03T16:26:27.9338671Z FAILED [0.0079s] [ 0%] 2024-04-03T16:26:27.9338691Z 2024-04-03T16:26:27.9338947Z ==================================== RERUNS ==================================== 2024-04-03T16:26:27.9339394Z __ TestForeachCUDA.test_binary_op_list_error_cases__foreach_add_cuda_float16 ___ 2024-04-03T16:26:27.9339572Z Traceback (most recent call last): 2024-04-03T16:26:27.9340222Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 438, in test_binary_op_list_error_cases 2024-04-03T16:26:27.9340407Z foreach_op(tensors1, tensors2) 2024-04-03T16:26:27.9341185Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9342046Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9342215Z C++ CapturedTraceback: 2024-04-03T16:26:27.9342992Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9343984Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9344414Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9344889Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9345289Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9346173Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9346803Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9347597Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9350882Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9351516Z #13 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9352301Z #14 at::native::foreach_tensor_add_list_kernel_slow(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9353066Z #15 at::native::foreach_tensor_add_list_kernel_cuda(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9354150Z #16 at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9358739Z #17 c10::impl::wrap_kernel_functor_unboxed_ > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9359810Z #18 at::_ops::_foreach_add_List::redispatch(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9360988Z #19 torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.9365938Z #20 c10::impl::wrap_kernel_functor_unboxed_ > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.9366617Z #21 at::_ops::_foreach_add_List::call(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9367243Z #22 torch::autograd::THPVariable__foreach_add(_object*, _object*, _object*) from python_torch_functions_1.cpp:0 2024-04-03T16:26:27.9367410Z #23 cfunction_call from :0 2024-04-03T16:26:27.9367632Z #24 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9367964Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9368167Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9368330Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9368533Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9368711Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9368871Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9369071Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9369248Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9369428Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9369613Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9369793Z #35 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9369963Z #36 PyObject_Call from ??:0 2024-04-03T16:26:27.9370154Z #37 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9370462Z #38 method_vectorcall from :0 2024-04-03T16:26:27.9370659Z #39 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9370819Z #40 method_vectorcall from :0 2024-04-03T16:26:27.9371012Z #41 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9371187Z #42 method_vectorcall from :0 2024-04-03T16:26:27.9371375Z #43 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9371550Z #44 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9371752Z #45 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9371916Z #46 method_vectorcall from :0 2024-04-03T16:26:27.9372106Z #47 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9372279Z #48 method_vectorcall from :0 2024-04-03T16:26:27.9372466Z #49 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9372623Z #50 method_vectorcall from :0 2024-04-03T16:26:27.9372829Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9372987Z #52 method_vectorcall from :0 2024-04-03T16:26:27.9373142Z #53 PyObject_Call from ??:0 2024-04-03T16:26:27.9373354Z #54 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9373535Z #55 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9373828Z #56 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9374014Z #57 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9374163Z #58 slot_tp_call from :0 2024-04-03T16:26:27.9374392Z #59 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9374583Z #60 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9374759Z #61 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9374960Z #62 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9375139Z #63 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9375328Z #64 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9375519Z #65 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9375712Z #66 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9375872Z #67 method_vectorcall from :0 2024-04-03T16:26:27.9376070Z #68 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9376257Z #69 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9376528Z #70 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9376724Z #71 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9376874Z #72 slot_tp_call from :0 2024-04-03T16:26:27.9377043Z #73 PyObject_Call from ??:0 2024-04-03T16:26:27.9377244Z #74 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9377423Z #75 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9377623Z #76 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9377786Z #77 method_vectorcall from :0 2024-04-03T16:26:27.9377974Z #78 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9378159Z #79 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9378347Z #80 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9378522Z #81 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9378721Z #82 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9378902Z #83 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9379088Z #84 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9379290Z #85 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9379480Z #86 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9379658Z #87 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9379944Z #88 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9380119Z #89 method_vectorcall from :0 2024-04-03T16:26:27.9380306Z #90 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9380496Z #91 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9380767Z #92 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9380946Z #93 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9381110Z #94 slot_tp_call from :0 2024-04-03T16:26:27.9381332Z #95 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9381534Z #96 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9381708Z #97 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9381889Z #98 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9382087Z #99 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9382381Z #100 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9382547Z #101 method_vectorcall from :0 2024-04-03T16:26:27.9382748Z #102 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9382931Z #103 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9383215Z #104 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9383413Z #105 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9383567Z #106 slot_tp_call from :0 2024-04-03T16:26:27.9383796Z #107 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9384003Z #108 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9384187Z #109 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9384377Z #110 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9384570Z #111 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9384756Z #112 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9384940Z #113 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9385139Z #114 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9385330Z #115 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9385539Z #116 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9385699Z #117 method_vectorcall from :0 2024-04-03T16:26:27.9385889Z #118 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9386089Z #119 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9386366Z #120 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9386552Z #121 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9386710Z #122 slot_tp_call from :0 2024-04-03T16:26:27.9386936Z #123 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9387130Z #124 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9387320Z #125 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9387506Z #126 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9387687Z #127 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9387893Z #128 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9388049Z #129 _PyEval_Vector from :0 2024-04-03T16:26:27.9388228Z #130 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9388396Z #131 run_eval_code_obj from :0 2024-04-03T16:26:27.9388539Z #132 run_mod from :0 2024-04-03T16:26:27.9388698Z #133 pyrun_file.cold from :0 2024-04-03T16:26:27.9388965Z #134 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9389193Z #135 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9389388Z #136 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9389548Z #137 Py_BytesMain from ??:0 2024-04-03T16:26:27.9389713Z #138 __libc_start_main from ??:0 2024-04-03T16:26:27.9389870Z #139 _start from ??:0 2024-04-03T16:26:27.9389893Z 2024-04-03T16:26:27.9389912Z 2024-04-03T16:26:27.9390302Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.9390323Z 2024-04-03T16:26:27.9391090Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9391949Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9392115Z C++ CapturedTraceback: 2024-04-03T16:26:27.9392888Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9393978Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9394419Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9394897Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9395285Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9396061Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9396688Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9397590Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9401052Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9401748Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9402730Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9406264Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9406776Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9407309Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9407471Z #18 cfunction_call from :0 2024-04-03T16:26:27.9407694Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9407883Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9408063Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9408254Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9408421Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9408587Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9408766Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9408935Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9409103Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9409282Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9409448Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9409610Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9409787Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9409960Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9410124Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9410310Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9410464Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9410652Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9410940Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9411143Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9411356Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9411532Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9411698Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9411892Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9412050Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9412226Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9412395Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9412571Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9412739Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9412916Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9413066Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9413358Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9413534Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9413701Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9413975Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9414148Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9414292Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9414510Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9414689Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9414861Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9415050Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9415221Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9415397Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9415576Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9415750Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9415907Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9416101Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9416267Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9416532Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9416710Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9416857Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9417022Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9417200Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9417365Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9417552Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9417709Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9417886Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9418069Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9418245Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9418414Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9418600Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9418765Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9418947Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9419124Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9419296Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9419471Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9419657Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9419811Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9419985Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9420168Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9420423Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9420608Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9420753Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9420959Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9421144Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9421314Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9421496Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9421692Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9421867Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9422027Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9422318Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9422511Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9422779Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9422967Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9423110Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9423321Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9423521Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9423698Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9423874Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9424060Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9424234Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9424421Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9424698Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9424866Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9425053Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9425212Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9425394Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9425582Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9425847Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9426019Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9426185Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9426394Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9426572Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9426759Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9426935Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9427108Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9427297Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9427457Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9427615Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9427777Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9427915Z #129 run_mod from :0 2024-04-03T16:26:27.9428095Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9428330Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9428541Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9428726Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9428881Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9429039Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9429198Z #136 _start from ??:0 2024-04-03T16:26:27.9429220Z 2024-04-03T16:26:27.9429239Z 2024-04-03T16:26:27.9429606Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.9429627Z 2024-04-03T16:26:27.9429802Z Traceback (most recent call last): 2024-04-03T16:26:27.9430420Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 440, in test_binary_op_list_error_cases 2024-04-03T16:26:27.9430734Z with self.assertRaisesRegex(type(e), re.escape(str(e))): 2024-04-03T16:26:27.9431222Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 239, in __exit__ 2024-04-03T16:26:27.9431575Z self._raiseFailure('"{}" does not match "{}"'.format( 2024-04-03T16:26:27.9432092Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 163, in _raiseFailure 2024-04-03T16:26:27.9432330Z raise self.test_case.failureException(msg) 2024-04-03T16:26:27.9433234Z AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.9434153Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.9434333Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.9435136Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9436319Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9436794Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9437300Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9437729Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9438562Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9439480Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9440351Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9443946Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9444572Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9445365Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9446163Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9447287Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9451888Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9452788Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9453990Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9459010Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9459769Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9460446Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.9460625Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.9460858Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9461213Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9461419Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9461589Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9461800Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9462006Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9462174Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9462392Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9462585Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9462746Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9462960Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9463150Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9463315Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9463538Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9463705Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9463917Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9464098Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9464306Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9464478Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9464699Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9464892Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9465095Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9465270Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9465477Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9465640Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9465858Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9466021Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9466242Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9466410Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9466574Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9466803Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9466996Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9467276Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9467487Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9467648Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9467876Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9468097Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9468289Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9468492Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9468699Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9468903Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9469096Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9469312Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9469477Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9469703Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9469893Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9470166Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9470458Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9470630Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9470792Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9471013Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9471206Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9471409Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9471588Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9471789Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9471982Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9472199Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9472391Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9472710Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9472908Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9473108Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9473324Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9473526Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9473711Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9473925Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9474088Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9474294Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9474500Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9474777Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9474969Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9475144Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9475373Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9475588Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9475793Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9475996Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9476188Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9476423Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9476594Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9476821Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9477022Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9477309Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9477523Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9477681Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9477913Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9478139Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9478334Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9478555Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9478767Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9478985Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9479371Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9479604Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9479806Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9480028Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9480198Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9480408Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9480624Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9480910Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9481112Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9481285Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9481528Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9481784Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9481997Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9482328Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9482546Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9482771Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9482933Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.9483105Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.9483281Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.9483432Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.9483594Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.9483858Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9484088Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9484286Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.9484448Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.9484742Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.9484911Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.9485662Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9486467Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9486637Z C++ CapturedTraceback: 2024-04-03T16:26:27.9487345Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9488271Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9488676Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9489118Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9489496Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9490226Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9490800Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9491555Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9494605Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9495301Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9496265Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9499788Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9500387Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9500949Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9501102Z #18 cfunction_call from :0 2024-04-03T16:26:27.9501309Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9501511Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9501684Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9501862Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9502044Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9502200Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9502377Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9502554Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9502811Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9502988Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9503172Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9503327Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9503530Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9503697Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9503853Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9504040Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9504197Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9504367Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9504528Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9504705Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9504860Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9505052Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9505221Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9505395Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9505574Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9505746Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9505904Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9506086Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9506247Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9506422Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9506592Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9506741Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9506913Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9507096Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9507355Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9507539Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9507687Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9507895Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9508084Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9508253Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9508437Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9508617Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9508794Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9508965Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9509153Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9509311Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9509484Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9509661Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9509912Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9510079Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9510231Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9510385Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9510563Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9510739Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9510914Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9511080Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9511319Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9511492Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9511684Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9511951Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9512140Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9512323Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9512497Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9512665Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9512854Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9513021Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9513193Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9513361Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9513531Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9513695Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9513965Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9514234Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9514378Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9514595Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9514779Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9514960Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9515139Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9515306Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9515490Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9515646Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9515822Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9516007Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9516275Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9516449Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9516607Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9516821Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9517014Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9517202Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9517380Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9517549Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9517743Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9517919Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9518133Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9518314Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9518491Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9518662Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9518838Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9519122Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9519480Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9519648Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9519793Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9520025Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9520206Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9520375Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9520568Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9520747Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9520923Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9521088Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9521243Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9521392Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9521537Z #129 run_mod from :0 2024-04-03T16:26:27.9521694Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9521932Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9522169Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9522339Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9522490Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9522662Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9522810Z #136 _start from ??:0 2024-04-03T16:26:27.9522941Z " 2024-04-03T16:26:27.9522962Z 2024-04-03T16:26:27.9523292Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.9524028Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.9524052Z 2024-04-03T16:26:27.9524493Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.9524916Z __ TestForeachCUDA.test_binary_op_list_error_cases__foreach_add_cuda_float16 ___ 2024-04-03T16:26:27.9525087Z Traceback (most recent call last): 2024-04-03T16:26:27.9525701Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 438, in test_binary_op_list_error_cases 2024-04-03T16:26:27.9525868Z foreach_op(tensors1, tensors2) 2024-04-03T16:26:27.9526587Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9527381Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9527672Z C++ CapturedTraceback: 2024-04-03T16:26:27.9528414Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9529335Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9529734Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9530186Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9530547Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9531281Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9531848Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9532596Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9535676Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9536242Z #13 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9536975Z #14 at::native::foreach_tensor_add_list_kernel_slow(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9537698Z #15 at::native::foreach_tensor_add_list_kernel_cuda(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9538721Z #16 at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9542971Z #17 c10::impl::wrap_kernel_functor_unboxed_ > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9543891Z #18 at::_ops::_foreach_add_List::redispatch(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9544999Z #19 torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.9549594Z #20 c10::impl::wrap_kernel_functor_unboxed_ > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.9550321Z #21 at::_ops::_foreach_add_List::call(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9550908Z #22 torch::autograd::THPVariable__foreach_add(_object*, _object*, _object*) from python_torch_functions_1.cpp:0 2024-04-03T16:26:27.9551075Z #23 cfunction_call from :0 2024-04-03T16:26:27.9551285Z #24 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9551463Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9551644Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9551806Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9551981Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9552168Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9552326Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9552502Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9552681Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9552829Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9553007Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9553186Z #35 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9553339Z #36 PyObject_Call from ??:0 2024-04-03T16:26:27.9553516Z #37 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9553692Z #38 method_vectorcall from :0 2024-04-03T16:26:27.9553865Z #39 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9554034Z #40 method_vectorcall from :0 2024-04-03T16:26:27.9554210Z #41 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9554365Z #42 method_vectorcall from :0 2024-04-03T16:26:27.9554562Z #43 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9554733Z #44 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9554905Z #45 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9555074Z #46 method_vectorcall from :0 2024-04-03T16:26:27.9555246Z #47 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9555399Z #48 method_vectorcall from :0 2024-04-03T16:26:27.9555583Z #49 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9555730Z #50 method_vectorcall from :0 2024-04-03T16:26:27.9555901Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9556065Z #52 method_vectorcall from :0 2024-04-03T16:26:27.9556216Z #53 PyObject_Call from ??:0 2024-04-03T16:26:27.9556390Z #54 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9556578Z #55 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9556835Z #56 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9557000Z #57 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9557155Z #58 slot_tp_call from :0 2024-04-03T16:26:27.9557370Z #59 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9557545Z #60 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9557722Z #61 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9557988Z #62 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9558184Z #63 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9558355Z #64 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9558522Z #65 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9558707Z #66 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9558861Z #67 method_vectorcall from :0 2024-04-03T16:26:27.9559124Z #68 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9559407Z #69 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9559651Z #70 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9559814Z #71 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9559971Z #72 slot_tp_call from :0 2024-04-03T16:26:27.9560122Z #73 PyObject_Call from ??:0 2024-04-03T16:26:27.9560433Z #74 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9560615Z #75 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9560789Z #76 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9560940Z #77 method_vectorcall from :0 2024-04-03T16:26:27.9561128Z #78 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9561293Z #79 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9561467Z #80 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9561646Z #81 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9561816Z #82 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9561994Z #83 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9562171Z #84 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9562338Z #85 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9562525Z #86 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9562696Z #87 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9562866Z #88 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9563029Z #89 method_vectorcall from :0 2024-04-03T16:26:27.9563210Z #90 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9563377Z #91 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9563642Z #92 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9563813Z #93 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9563959Z #94 slot_tp_call from :0 2024-04-03T16:26:27.9564175Z #95 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9564347Z #96 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9564513Z #97 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9564695Z #98 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9564859Z #99 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9565040Z #100 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9565210Z #101 method_vectorcall from :0 2024-04-03T16:26:27.9565389Z #102 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9565576Z #103 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9565840Z #104 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9566019Z #105 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9566171Z #106 slot_tp_call from :0 2024-04-03T16:26:27.9566384Z #107 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9566566Z #108 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9566752Z #109 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9566926Z #110 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9567097Z #111 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9567279Z #112 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9567450Z #113 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9567625Z #114 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9567804Z #115 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9567979Z #116 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9568138Z #117 method_vectorcall from :0 2024-04-03T16:26:27.9568326Z #118 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9568495Z #119 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9568763Z #120 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9568944Z #121 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9569088Z #122 slot_tp_call from :0 2024-04-03T16:26:27.9569451Z #123 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9569651Z #124 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9569822Z #125 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9570005Z #126 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9570173Z #127 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9570348Z #128 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9570506Z #129 _PyEval_Vector from :0 2024-04-03T16:26:27.9570664Z #130 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9570812Z #131 run_eval_code_obj from :0 2024-04-03T16:26:27.9570963Z #132 run_mod from :0 2024-04-03T16:26:27.9571117Z #133 pyrun_file.cold from :0 2024-04-03T16:26:27.9571354Z #134 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9571578Z #135 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9571867Z #136 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9572018Z #137 Py_BytesMain from ??:0 2024-04-03T16:26:27.9572189Z #138 __libc_start_main from ??:0 2024-04-03T16:26:27.9572337Z #139 _start from ??:0 2024-04-03T16:26:27.9572359Z 2024-04-03T16:26:27.9572379Z 2024-04-03T16:26:27.9572762Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.9572783Z 2024-04-03T16:26:27.9573499Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9574294Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9574459Z C++ CapturedTraceback: 2024-04-03T16:26:27.9575170Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9576093Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9576506Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9576946Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9577317Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9578039Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9578605Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9579357Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9582407Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9583093Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9584055Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9587665Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9588188Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9588729Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9588883Z #18 cfunction_call from :0 2024-04-03T16:26:27.9589093Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9589284Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9589455Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9589627Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9589904Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9590054Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9590229Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9590412Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9590565Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9590739Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9590917Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9591068Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9591242Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9591419Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9591569Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9591751Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9591903Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9592074Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9592243Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9592410Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9592570Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9592752Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9592916Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9593090Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9593252Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9593421Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9593571Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9593756Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9593909Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9594080Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9594244Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9594395Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9594566Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9594744Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9595001Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9595192Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9595338Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9595543Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9595733Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9595906Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9596082Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9596259Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9596430Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9596598Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9596781Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9596934Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9597110Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9597289Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9597538Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9597706Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9597861Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9598022Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9598196Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9598371Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9598632Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9598815Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9598985Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9599333Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9599523Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9599682Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9599851Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9600030Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9600204Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9600367Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9600553Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9600717Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9601040Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9601211Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9601384Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9601551Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9601825Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9601988Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9602128Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9602344Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9602519Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9602683Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9602872Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9603035Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9603221Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9603372Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9603544Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9603728Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9604003Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9604176Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9604333Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9604550Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9604732Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9604917Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9605094Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9605263Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9605447Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9605619Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9605794Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9605980Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9606153Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9606309Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9606497Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9606680Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9606951Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9607124Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9607271Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9607493Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9607670Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9607836Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9608019Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9608191Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9608368Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9608526Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9608681Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9608834Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9608981Z #129 run_mod from :0 2024-04-03T16:26:27.9609129Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9609361Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9609592Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9609762Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9609911Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9610201Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9610359Z #136 _start from ??:0 2024-04-03T16:26:27.9610379Z 2024-04-03T16:26:27.9610398Z 2024-04-03T16:26:27.9610779Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.9610800Z 2024-04-03T16:26:27.9610965Z Traceback (most recent call last): 2024-04-03T16:26:27.9611618Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 440, in test_binary_op_list_error_cases 2024-04-03T16:26:27.9611925Z with self.assertRaisesRegex(type(e), re.escape(str(e))): 2024-04-03T16:26:27.9612404Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 239, in __exit__ 2024-04-03T16:26:27.9612751Z self._raiseFailure('"{}" does not match "{}"'.format( 2024-04-03T16:26:27.9613385Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 163, in _raiseFailure 2024-04-03T16:26:27.9613610Z raise self.test_case.failureException(msg) 2024-04-03T16:26:27.9614529Z AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.9615448Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.9615611Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.9616426Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9617486Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9617947Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9618468Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9618891Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9619731Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9620404Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9621268Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9624728Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9625358Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9626154Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9626931Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9628055Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9632741Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9633733Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9634937Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9640040Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9640775Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9641461Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.9641625Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.9641867Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9642076Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9642273Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9642453Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9642663Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9642863Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9643038Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9643238Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9643432Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9643603Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9643806Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9643996Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9644170Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9644375Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9644648Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9644852Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9644915Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9645122Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9645282Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9645483Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9645695Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9645895Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9646056Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9646401Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9646586Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9646788Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9646960Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9647157Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9647314Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9647484Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9647686Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9647889Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9648173Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9648370Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9648657Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9648884Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9649087Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9649294Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9649493Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9649681Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9649892Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9650080Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9650282Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9650459Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9650657Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9650845Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9651127Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9651321Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9651490Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9651654Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9651854Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9652060Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9652259Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9652420Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9652635Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9652823Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9653019Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9653219Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9653413Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9653599Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9653807Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9653998Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9654203Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9654396Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9654601Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9654786Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9654988Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9655173Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9655548Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9655736Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9655798Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9656029Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9656230Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9656421Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9656630Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9656817Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9657045Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9657229Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9657436Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9657718Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9658027Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9658226Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9658393Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9658623Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9658829Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9659037Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9659242Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9659434Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9659656Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9659852Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9660160Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9660373Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9660578Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9660749Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9660964Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9661161Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9661439Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9661646Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9661802Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9662038Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9662242Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9662433Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9662650Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9662845Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9663061Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9663234Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.9663404Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.9663570Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.9663729Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.9663893Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.9664146Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9664384Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9664566Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.9664726Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.9664907Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.9665056Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.9665808Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9666609Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9666773Z C++ CapturedTraceback: 2024-04-03T16:26:27.9667494Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9668417Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9668806Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9669254Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9669612Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9670341Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9670919Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9671753Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9674834Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9675503Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9676595Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9680295Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9680812Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9681350Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9681510Z #18 cfunction_call from :0 2024-04-03T16:26:27.9681714Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9681902Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9682079Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9682255Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9682419Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9682577Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9682749Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9682915Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9683074Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9683249Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9683412Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9683575Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9683749Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9683929Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9684090Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9684264Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9684419Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9684612Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9684769Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9684955Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9685109Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9685283Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9685466Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9685635Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9685787Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9685964Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9686116Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9686284Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9686447Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9686628Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9686783Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9686944Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9687116Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9687446Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9687740Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9687906Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9688047Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9688263Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9688442Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9688607Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9688789Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9688953Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9689143Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9689309Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9689482Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9689770Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9689941Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9690107Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9690373Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9690538Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9690681Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9690847Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9691022Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9691187Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9691377Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9691527Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9691696Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9691876Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9692050Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9692219Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9692412Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9692579Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9692763Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9692928Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9693104Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9693285Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9693460Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9693612Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9693797Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9693958Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9694209Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9694391Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9694536Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9694736Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9694921Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9695089Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9695269Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9695448Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9695617Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9695777Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9695968Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9696143Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9696419Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9696589Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9696735Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9696951Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9697130Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9697301Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9697490Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9697662Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9697836Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9698028Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9698205Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9698371Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9698647Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9698818Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9698991Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9699177Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9699439Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9699611Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9699763Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9699974Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9700162Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9700329Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9700503Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9700686Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9700953Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9701102Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9701272Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9701422Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9701563Z #129 run_mod from :0 2024-04-03T16:26:27.9701728Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9701960Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9702174Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9702352Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9702503Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9702659Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9702809Z #136 _start from ??:0 2024-04-03T16:26:27.9702928Z " 2024-04-03T16:26:27.9702949Z 2024-04-03T16:26:27.9703279Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.9703892Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.9703921Z 2024-04-03T16:26:27.9704330Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.9704585Z =================================== FAILURES =================================== 2024-04-03T16:26:27.9705012Z __ TestForeachCUDA.test_binary_op_list_error_cases__foreach_add_cuda_float16 ___ 2024-04-03T16:26:27.9705184Z Traceback (most recent call last): 2024-04-03T16:26:27.9705792Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 438, in test_binary_op_list_error_cases 2024-04-03T16:26:27.9705961Z foreach_op(tensors1, tensors2) 2024-04-03T16:26:27.9706656Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9707462Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9707617Z C++ CapturedTraceback: 2024-04-03T16:26:27.9708339Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9709274Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9709672Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9710126Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9710485Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9711198Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9711836Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9712579Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9715742Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9716253Z #13 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9716975Z #14 at::native::foreach_tensor_add_list_kernel_slow(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9717680Z #15 at::native::foreach_tensor_add_list_kernel_cuda(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9718791Z #16 at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9723238Z #17 c10::impl::wrap_kernel_functor_unboxed_ > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA_List__foreach_add>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9724056Z #18 at::_ops::_foreach_add_List::redispatch(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9725146Z #19 torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List(c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.9729730Z #20 c10::impl::wrap_kernel_functor_unboxed_ > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&), &torch::autograd::VariableType::(anonymous namespace)::_foreach_add_List>, std::vector >, c10::guts::typelist::typelist, c10::ArrayRef, c10::Scalar const&> >, std::vector > (c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from VariableType_0.cpp:0 2024-04-03T16:26:27.9730362Z #21 at::_ops::_foreach_add_List::call(c10::ArrayRef, c10::ArrayRef, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9730963Z #22 torch::autograd::THPVariable__foreach_add(_object*, _object*, _object*) from python_torch_functions_1.cpp:0 2024-04-03T16:26:27.9731118Z #23 cfunction_call from :0 2024-04-03T16:26:27.9731324Z #24 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9731517Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9731687Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9731843Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9732031Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9732201Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9732362Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9732554Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9732722Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9733014Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9733234Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9733399Z #35 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9733550Z #36 PyObject_Call from ??:0 2024-04-03T16:26:27.9733736Z #37 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9733890Z #38 method_vectorcall from :0 2024-04-03T16:26:27.9734060Z #39 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9734221Z #40 method_vectorcall from :0 2024-04-03T16:26:27.9734392Z #41 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9734553Z #42 method_vectorcall from :0 2024-04-03T16:26:27.9734722Z #43 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9734889Z #44 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9735073Z #45 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9735350Z #46 method_vectorcall from :0 2024-04-03T16:26:27.9735523Z #47 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9735688Z #48 method_vectorcall from :0 2024-04-03T16:26:27.9735869Z #49 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9736022Z #50 method_vectorcall from :0 2024-04-03T16:26:27.9736203Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9736354Z #52 method_vectorcall from :0 2024-04-03T16:26:27.9736505Z #53 PyObject_Call from ??:0 2024-04-03T16:26:27.9736690Z #54 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9736858Z #55 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9737116Z #56 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9737294Z #57 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9737438Z #58 slot_tp_call from :0 2024-04-03T16:26:27.9737644Z #59 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9737829Z #60 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9737992Z #61 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9738185Z #62 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9738353Z #63 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9738521Z #64 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9738703Z #65 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9738877Z #66 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9739032Z #67 method_vectorcall from :0 2024-04-03T16:26:27.9739213Z #68 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9739377Z #69 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9739629Z #70 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9739812Z #71 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9739958Z #72 slot_tp_call from :0 2024-04-03T16:26:27.9740109Z #73 PyObject_Call from ??:0 2024-04-03T16:26:27.9740291Z #74 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9740459Z #75 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9740630Z #76 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9740804Z #77 method_vectorcall from :0 2024-04-03T16:26:27.9740978Z #78 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9741146Z #79 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9741331Z #80 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9741502Z #81 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9741675Z #82 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9741851Z #83 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9742022Z #84 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9742205Z #85 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9742378Z #86 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9742547Z #87 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9742728Z #88 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9742880Z #89 method_vectorcall from :0 2024-04-03T16:26:27.9743052Z #90 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9743228Z #91 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9743478Z #92 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9743655Z #93 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9743816Z #94 slot_tp_call from :0 2024-04-03T16:26:27.9744016Z #95 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9744278Z #96 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9744474Z #97 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9744646Z #98 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9744811Z #99 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9745003Z #100 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9745158Z #101 method_vectorcall from :0 2024-04-03T16:26:27.9745333Z #102 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9745522Z #103 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9745787Z #104 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9745973Z #105 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9746116Z #106 slot_tp_call from :0 2024-04-03T16:26:27.9746327Z #107 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9746625Z #108 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9746793Z #109 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9746971Z #110 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9747156Z #111 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9747332Z #112 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9747503Z #113 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9747688Z #114 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9747854Z #115 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9748029Z #116 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9748200Z #117 method_vectorcall from :0 2024-04-03T16:26:27.9748372Z #118 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9748541Z #119 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9748811Z #120 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9748982Z #121 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9749126Z #122 slot_tp_call from :0 2024-04-03T16:26:27.9749362Z #123 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9749537Z #124 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9749714Z #125 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9749898Z #126 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9750068Z #127 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9750258Z #128 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9750411Z #129 _PyEval_Vector from :0 2024-04-03T16:26:27.9750568Z #130 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9750729Z #131 run_eval_code_obj from :0 2024-04-03T16:26:27.9750864Z #132 run_mod from :0 2024-04-03T16:26:27.9751019Z #133 pyrun_file.cold from :0 2024-04-03T16:26:27.9751263Z #134 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9751477Z #135 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9751649Z #136 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9751815Z #137 Py_BytesMain from ??:0 2024-04-03T16:26:27.9751973Z #138 __libc_start_main from ??:0 2024-04-03T16:26:27.9752126Z #139 _start from ??:0 2024-04-03T16:26:27.9752149Z 2024-04-03T16:26:27.9752168Z 2024-04-03T16:26:27.9752548Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.9752569Z 2024-04-03T16:26:27.9753293Z RuntimeError: The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9754114Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9754270Z C++ CapturedTraceback: 2024-04-03T16:26:27.9754978Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9755910Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9756306Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9756763Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9757123Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9757928Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9758529Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9759453Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9762529Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9763355Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9764317Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9767848Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9768375Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9768903Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9769052Z #18 cfunction_call from :0 2024-04-03T16:26:27.9769268Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9769451Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9769623Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9769813Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9769983Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9770138Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9770319Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9770499Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9770650Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9770837Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9771007Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9771154Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9771338Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9771507Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9771670Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9771844Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9771998Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9772180Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9772334Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9772508Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9772667Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9772840Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9773008Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9773200Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9773353Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9773529Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9773797Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9773989Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9774141Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9774319Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9774470Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9774617Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9774794Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9774959Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9775214Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9775397Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9775539Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9775752Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9776048Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9776212Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9776394Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9776561Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9776737Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9776917Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9777087Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9777239Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9777422Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9777586Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9777837Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9778018Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9778167Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9778319Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9778508Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9778669Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9778855Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9779021Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9779193Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9779382Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9779559Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9779726Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9779910Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9780076Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9780248Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9780432Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9780605Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9780770Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9780958Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9781111Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9781279Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9781453Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9781716Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9781886Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9782044Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9782248Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9782422Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9782599Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9782768Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9782946Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9783118Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9783270Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9783448Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9783626Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9783893Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9784073Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9784218Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9784437Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9784628Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9784799Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9785065Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9785269Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9785443Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9785611Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9785800Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9785968Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9786141Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9786308Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9786485Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9786661Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9786928Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9787098Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9787390Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9787600Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9787774Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9787956Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9788130Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9788304Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9788493Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9788644Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9788803Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9788963Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9789098Z #129 run_mod from :0 2024-04-03T16:26:27.9789252Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9789497Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9789710Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9789881Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9790059Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9790215Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9790355Z #136 _start from ??:0 2024-04-03T16:26:27.9790392Z 2024-04-03T16:26:27.9790411Z 2024-04-03T16:26:27.9790787Z During handling of the above exception, another exception occurred: 2024-04-03T16:26:27.9790808Z 2024-04-03T16:26:27.9790978Z Traceback (most recent call last): 2024-04-03T16:26:27.9791587Z File "/var/lib/jenkins/workspace/test/test_foreach.py", line 440, in test_binary_op_list_error_cases 2024-04-03T16:26:27.9791888Z with self.assertRaisesRegex(type(e), re.escape(str(e))): 2024-04-03T16:26:27.9792362Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 239, in __exit__ 2024-04-03T16:26:27.9792721Z self._raiseFailure('"{}" does not match "{}"'.format( 2024-04-03T16:26:27.9793230Z File "/opt/conda/envs/py_3.10/lib/python3.10/unittest/case.py", line 163, in _raiseFailure 2024-04-03T16:26:27.9793459Z raise self.test_case.failureException(msg) 2024-04-03T16:26:27.9794372Z AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.9795292Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.9795467Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.9796261Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9797324Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9797786Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9798300Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9798738Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9799877Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9800590Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9801460Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9804894Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9805638Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9806431Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9807211Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9808324Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9812969Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9813869Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9815075Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9820006Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9820734Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9821504Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.9821700Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.9821935Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9822145Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9822349Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9822517Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9822727Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9822934Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9823094Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9823297Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9823607Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9823766Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9823969Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9824169Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9824332Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9824536Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9824713Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9824915Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9825084Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9825283Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9825445Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9825655Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9825847Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9826043Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9826228Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9826432Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9826594Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9826814Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9826983Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9827186Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9827359Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9827521Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9827720Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9827917Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9828197Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9828398Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9828551Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9828774Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9828988Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9829189Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9829390Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9829587Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9829793Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9829981Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9830199Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9830365Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9830562Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9830760Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9831031Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9831217Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9831385Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9831544Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9831753Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9831957Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9832162Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9832331Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9832642Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9832852Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9833068Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9833254Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9833453Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9833656Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9833859Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9834047Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9834256Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9834442Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9834638Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9834913Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9835117Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9835320Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9835601Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9835792Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9835957Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9836241Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9836436Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9836635Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9836835Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9837021Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9837246Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9837411Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9837625Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9837848Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9838125Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9838322Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9838494Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9838727Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9838946Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9839322Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9839529Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9839727Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9839933Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9840128Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9840346Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9840542Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9840753Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9840948Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9841157Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9841347Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9841652Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9841854Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9842009Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9842251Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9842460Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9842669Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9842881Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9843074Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9843294Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9843456Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.9843635Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.9843812Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.9843960Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.9844116Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.9844504Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9844752Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9844937Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.9845115Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.9845287Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.9845432Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.9846178Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9846982Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9847144Z C++ CapturedTraceback: 2024-04-03T16:26:27.9847992Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9848925Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9849331Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9849772Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9850132Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9850861Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9851431Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9852176Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9855241Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9855920Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9856879Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9860414Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9860910Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9861437Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9861600Z #18 cfunction_call from :0 2024-04-03T16:26:27.9861805Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9862002Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9862176Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9862349Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9862612Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9862779Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9862953Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9863128Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9863278Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9863448Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9863621Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9863771Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9863941Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9864119Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9864269Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9864444Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9864702Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9864872Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9865025Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9865205Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9865364Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9865551Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9865717Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9865892Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9866050Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9866222Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9866375Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9866564Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9866715Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9866884Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9867041Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9867191Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9867362Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9867550Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9867802Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9867966Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9868128Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9868338Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9868509Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9868687Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9868856Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9869021Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9869207Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9869375Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9869559Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9869712Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9869887Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9870063Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9870326Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9870494Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9870649Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9870807Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9870980Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9871157Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9871328Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9871478Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9871661Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9871824Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9871997Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9872170Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9872336Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9872502Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9872684Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9872860Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9873048Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9873216Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9873386Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9873637Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9873825Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9873987Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9874250Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9874411Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9874554Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9874768Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9874940Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9875104Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9875294Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9875459Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9875629Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9875916Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9876088Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9876262Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9876547Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9876715Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9876869Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9877085Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9877263Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9877442Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9877618Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9877787Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9877975Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9878144Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9878324Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9878510Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9878694Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9878853Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9879139Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9879414Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9879665Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9879845Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9879988Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9880198Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9880389Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9880560Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9880748Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9880918Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9881089Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9881249Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9881410Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9881576Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9881726Z #129 run_mod from :0 2024-04-03T16:26:27.9881880Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9882114Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9882345Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9882510Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9882658Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9882823Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9882961Z #136 _start from ??:0 2024-04-03T16:26:27.9883079Z " 2024-04-03T16:26:27.9883101Z 2024-04-03T16:26:27.9883445Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.9884040Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.9884061Z 2024-04-03T16:26:27.9884477Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.9885425Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_foreach/test_foreach-26a998c1bb36d87a.xml - 2024-04-03T16:26:27.9885711Z =========================== short test summary info ============================ 2024-04-03T16:26:27.9887490Z FAILED [0.0079s] test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16 - AssertionError: "The\ size\ of\ tensor\ a\ \(10\)\ must\ match\ the\ size\ of\ tensor\ b\ \(11\)\ at\ non\-singleton\ dimension\ 1\ 2024-04-03T16:26:27.9888429Z Exception\ raised\ from\ infer_size_impl\ at\ /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils\.cpp:31\ \(most\ recent\ call\ first\):\ 2024-04-03T16:26:27.9888607Z C\+\+\ CapturedTraceback:\ 2024-04-03T16:26:27.9889402Z \#4\ c10::Error::Error\(c10::SourceLocation,\ std::__cxx11::basic_string,\ std::allocator\ >\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9890463Z \#5\ c10::detail::torchCheckFail\(char\ const\*,\ char\ const\*,\ unsigned\ int,\ std::__cxx11::basic_string,\ std::allocator\ >\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9891059Z \#6\ at::infer_size_dimvector\(c10::ArrayRef,\ c10::ArrayRef\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9891576Z \#7\ at::TensorIteratorBase::compute_shape\(at::TensorIteratorConfig\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9891988Z \#8\ at::TensorIteratorBase::build\(at::TensorIteratorConfig\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9892824Z \#9\ at::TensorIteratorBase::build_borrowing_binary_op\(at::TensorBase\ const\&,\ at::TensorBase\ const\&,\ at::TensorBase\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9893501Z \#10\ at::meta::structured_add_Tensor::meta\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9894376Z \#11\ at::\(anonymous\ namespace\)::wrapper_CUDA_add_Tensor\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9897834Z \#12\ c10::impl::wrap_kernel_functor_unboxed_,\ at::Tensor,\ c10::guts::typelist::typelist\ >,\ at::Tensor\ \(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9898461Z \#13\ at::_ops::add_Tensor::call\(at::Tensor\ const\&,\ at::Tensor\ const\&,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9899254Z \#14\ at::native::foreach_tensor_add_list_kernel_slow\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9900028Z \#15\ at::native::foreach_tensor_add_list_kernel_cuda\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9901163Z \#16\ at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9905731Z \#17\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&at::\(anonymous\ namespace\)::\(anonymous\ namespace\)::wrapper_CUDA_List__foreach_add>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ RegisterCUDA\.cpp:0\ 2024-04-03T16:26:27.9906720Z \#18\ at::_ops::_foreach_add_List::redispatch\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9907934Z \#19\ torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List\(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9912920Z \#20\ c10::impl::wrap_kernel_functor_unboxed_\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\),\ \&torch::autograd::VariableType::\(anonymous\ namespace\)::_foreach_add_List>,\ std::vector\ >,\ c10::guts::typelist::typelist,\ c10::ArrayRef,\ c10::Scalar\ const\&>\ >,\ std::vector\ >\ \(c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)>::call\(c10::OperatorKernel\*,\ c10::DispatchKeySet,\ c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ VariableType_0\.cpp:0\ 2024-04-03T16:26:27.9913788Z \#21\ at::_ops::_foreach_add_List::call\(c10::ArrayRef,\ c10::ArrayRef,\ c10::Scalar\ const\&\)\ from\ \?\?:0\ 2024-04-03T16:26:27.9914463Z \#22\ torch::autograd::THPVariable__foreach_add\(_object\*,\ _object\*,\ _object\*\)\ from\ python_torch_functions_1\.cpp:0\ 2024-04-03T16:26:27.9914626Z \#23\ cfunction_call\ from\ :0\ 2024-04-03T16:26:27.9914866Z \#24\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9915084Z \#25\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9915281Z \#26\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9915468Z \#27\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9915678Z \#28\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9915885Z \#29\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9916050Z \#30\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9916257Z \#31\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9916459Z \#32\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9916619Z \#33\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9916825Z \#34\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9917025Z \#35\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9917187Z \#36\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9917394Z \#37\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9917570Z \#38\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9917774Z \#39\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9917937Z \#40\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9918153Z \#41\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9918329Z \#42\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9918527Z \#43\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9918731Z \#44\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9918937Z \#45\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9919279Z \#46\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9919493Z \#47\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9919654Z \#48\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9919866Z \#49\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9920026Z \#50\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9920224Z \#51\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9920393Z \#52\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9920555Z \#53\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9920754Z \#54\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9920953Z \#55\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9921230Z \#56\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9921429Z \#57\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9921598Z \#58\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9921949Z \#59\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9922176Z \#60\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9922379Z \#61\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9922577Z \#62\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9922778Z \#63\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9922979Z \#64\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9923168Z \#65\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9923382Z \#66\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9923545Z \#67\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9923745Z \#68\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9923946Z \#69\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9924345Z \#70\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9924533Z \#71\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9924698Z \#72\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9924861Z \#73\ PyObject_Call\ from\ \?\?:0\ 2024-04-03T16:26:27.9925068Z \#74\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9925268Z \#75\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9925470Z \#76\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9925634Z \#77\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9925848Z \#78\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9926037Z \#79\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9926248Z \#80\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9926438Z \#81\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9926640Z \#82\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9926842Z \#83\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9927043Z \#84\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9927240Z \#85\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9927449Z \#86\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9927638Z \#87\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9927842Z \#88\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9928018Z \#89\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9928218Z \#90\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9928409Z \#91\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9928703Z \#92\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9928894Z \#93\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9929049Z \#94\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9929284Z \#95\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9929486Z \#96\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9929688Z \#97\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9929891Z \#98\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9930093Z \#99\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9930323Z \#100\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9930489Z \#101\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9930706Z \#102\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9930914Z \#103\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9931192Z \#104\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9931387Z \#105\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9931556Z \#106\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9931788Z \#107\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9931999Z \#108\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9932210Z \#109\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9932414Z \#110\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9932612Z \#111\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9932848Z \#112\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9933046Z \#113\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9933266Z \#114\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9933564Z \#115\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9933791Z \#116\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9933971Z \#117\ method_vectorcall\ from\ :0\ 2024-04-03T16:26:27.9934175Z \#118\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9934368Z \#119\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9934667Z \#120\ _PyObject_FastCallDictTstate\.localalias\ from\ :0\ 2024-04-03T16:26:27.9934867Z \#121\ _PyObject_Call_Prepend\ from\ \?\?:0\ 2024-04-03T16:26:27.9935026Z \#122\ slot_tp_call\ from\ :0\ 2024-04-03T16:26:27.9935273Z \#123\ _PyObject_MakeTpCall\.localalias\ from\ :0\ 2024-04-03T16:26:27.9935477Z \#124\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9935676Z \#125\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9936018Z \#126\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9936217Z \#127\ _PyFunction_Vectorcall\ from\ \?\?:0\ 2024-04-03T16:26:27.9936424Z \#128\ _PyEval_EvalFrameDefault\ from\ \?\?:0\ 2024-04-03T16:26:27.9936601Z \#129\ _PyEval_Vector\ from\ :0\ 2024-04-03T16:26:27.9936775Z \#130\ PyEval_EvalCode\ from\ \?\?:0\ 2024-04-03T16:26:27.9936948Z \#131\ run_eval_code_obj\ from\ :0\ 2024-04-03T16:26:27.9937095Z \#132\ run_mod\ from\ :0\ 2024-04-03T16:26:27.9937256Z \#133\ pyrun_file\.cold\ from\ :0\ 2024-04-03T16:26:27.9937517Z \#134\ _PyRun_SimpleFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9937744Z \#135\ _PyRun_AnyFileObject\.localalias\ from\ :0\ 2024-04-03T16:26:27.9937930Z \#136\ Py_RunMain\.localalias\ from\ :0\ 2024-04-03T16:26:27.9938102Z \#137\ Py_BytesMain\ from\ \?\?:0\ 2024-04-03T16:26:27.9938273Z \#138\ __libc_start_main\ from\ \?\?:0\ 2024-04-03T16:26:27.9938424Z \#139\ _start\ from\ \?\?:0\ 2024-04-03T16:26:27.9939172Z " does not match "The size of tensor a (10) must match the size of tensor b (11) at non-singleton dimension 1 2024-04-03T16:26:27.9939983Z Exception raised from infer_size_impl at /var/lib/jenkins/workspace/aten/src/ATen/ExpandUtils.cpp:31 (most recent call first): 2024-04-03T16:26:27.9940147Z C++ CapturedTraceback: 2024-04-03T16:26:27.9940871Z #4 c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string, std::allocator >) from ??:0 2024-04-03T16:26:27.9941794Z #5 c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string, std::allocator > const&) from ??:0 2024-04-03T16:26:27.9942203Z #6 at::infer_size_dimvector(c10::ArrayRef, c10::ArrayRef) from ??:0 2024-04-03T16:26:27.9942645Z #7 at::TensorIteratorBase::compute_shape(at::TensorIteratorConfig const&) from ??:0 2024-04-03T16:26:27.9943006Z #8 at::TensorIteratorBase::build(at::TensorIteratorConfig&) from ??:0 2024-04-03T16:26:27.9943739Z #9 at::TensorIteratorBase::build_borrowing_binary_op(at::TensorBase const&, at::TensorBase const&, at::TensorBase const&) from ??:0 2024-04-03T16:26:27.9944318Z #10 at::meta::structured_add_Tensor::meta(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9945075Z #11 at::(anonymous namespace)::wrapper_CUDA_add_Tensor(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9948128Z #12 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from RegisterCUDA.cpp:0 2024-04-03T16:26:27.9948810Z #13 at::_ops::add_Tensor::redispatch(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9949871Z #14 torch::autograd::VariableType::(anonymous namespace)::add_Tensor(c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9953411Z #15 c10::impl::wrap_kernel_functor_unboxed_, at::Tensor, c10::guts::typelist::typelist >, at::Tensor (c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, at::Tensor const&, c10::Scalar const&) from VariableType_2.cpp:0 2024-04-03T16:26:27.9953924Z #16 at::_ops::add_Tensor::call(at::Tensor const&, at::Tensor const&, c10::Scalar const&) from ??:0 2024-04-03T16:26:27.9954549Z #17 torch::autograd::THPVariable_add(_object*, _object*, _object*) from python_torch_functions_2.cpp:0 2024-04-03T16:26:27.9954726Z #18 cfunction_call from :0 2024-04-03T16:26:27.9954937Z #19 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9955118Z #20 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9955300Z #21 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9955478Z #22 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9955648Z #23 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9955816Z #24 PyObject_Call from ??:0 2024-04-03T16:26:27.9955991Z #25 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9956172Z #26 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9956327Z #27 PyObject_Call from ??:0 2024-04-03T16:26:27.9956502Z #28 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9956677Z #29 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9956838Z #30 PyObject_Call from ??:0 2024-04-03T16:26:27.9957014Z #31 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9957195Z #32 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9957348Z #33 PyObject_Call from ??:0 2024-04-03T16:26:27.9957533Z #34 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9957703Z #35 method_vectorcall from :0 2024-04-03T16:26:27.9957875Z #36 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9958029Z #37 method_vectorcall from :0 2024-04-03T16:26:27.9958215Z #38 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9958367Z #39 method_vectorcall from :0 2024-04-03T16:26:27.9958537Z #40 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9958718Z #41 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9958892Z #42 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9959262Z #43 method_vectorcall from :0 2024-04-03T16:26:27.9959434Z #44 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9959594Z #45 method_vectorcall from :0 2024-04-03T16:26:27.9959757Z #46 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9959934Z #47 method_vectorcall from :0 2024-04-03T16:26:27.9960107Z #48 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9960278Z #49 method_vectorcall from :0 2024-04-03T16:26:27.9960438Z #50 PyObject_Call from ??:0 2024-04-03T16:26:27.9960616Z #51 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9960801Z #52 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9961055Z #53 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9961218Z #54 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9961382Z #55 slot_tp_call from :0 2024-04-03T16:26:27.9961583Z #56 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9971744Z #57 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9971971Z #58 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9972167Z #59 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9972364Z #60 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9972558Z #61 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9972748Z #62 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9972956Z #63 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9973119Z #64 method_vectorcall from :0 2024-04-03T16:26:27.9973312Z #65 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9973703Z #66 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9974009Z #67 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9974209Z #68 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9974361Z #69 slot_tp_call from :0 2024-04-03T16:26:27.9974524Z #70 PyObject_Call from ??:0 2024-04-03T16:26:27.9974730Z #71 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9974911Z #72 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9975100Z #73 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9975280Z #74 method_vectorcall from :0 2024-04-03T16:26:27.9975465Z #75 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9975645Z #76 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9975845Z #77 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9976172Z #78 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9976362Z #79 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9976557Z #80 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9976753Z #81 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9976935Z #82 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9977133Z #83 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9977312Z #84 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9977506Z #85 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9977680Z #86 method_vectorcall from :0 2024-04-03T16:26:27.9977867Z #87 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9978061Z #88 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9978334Z #89 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9978514Z #90 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9978679Z #91 slot_tp_call from :0 2024-04-03T16:26:27.9978896Z #92 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9979094Z #93 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9979291Z #94 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9979478Z #95 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9979659Z #96 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9979864Z #97 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9980027Z #98 method_vectorcall from :0 2024-04-03T16:26:27.9980220Z #99 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9980424Z #100 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9980705Z #101 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9980894Z #102 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9981065Z #103 slot_tp_call from :0 2024-04-03T16:26:27.9981291Z #104 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9981483Z #105 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9981687Z #106 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9981877Z #107 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9982092Z #108 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9982284Z #109 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9982471Z #110 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9982683Z #111 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9982873Z #112 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9983066Z #113 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9983246Z #114 method_vectorcall from :0 2024-04-03T16:26:27.9983438Z #115 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9983623Z #116 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9983919Z #117 _PyObject_FastCallDictTstate.localalias from :0 2024-04-03T16:26:27.9984108Z #118 _PyObject_Call_Prepend from ??:0 2024-04-03T16:26:27.9984265Z #119 slot_tp_call from :0 2024-04-03T16:26:27.9984513Z #120 _PyObject_MakeTpCall.localalias from :0 2024-04-03T16:26:27.9984707Z #121 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9984888Z #122 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9985099Z #123 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9985285Z #124 _PyFunction_Vectorcall from ??:0 2024-04-03T16:26:27.9985477Z #125 _PyEval_EvalFrameDefault from ??:0 2024-04-03T16:26:27.9985646Z #126 _PyEval_Vector from :0 2024-04-03T16:26:27.9985904Z #127 PyEval_EvalCode from ??:0 2024-04-03T16:26:27.9986089Z #128 run_eval_code_obj from :0 2024-04-03T16:26:27.9986234Z #129 run_mod from :0 2024-04-03T16:26:27.9986394Z #130 pyrun_file.cold from :0 2024-04-03T16:26:27.9986664Z #131 _PyRun_SimpleFileObject.localalias from :0 2024-04-03T16:26:27.9986892Z #132 _PyRun_AnyFileObject.localalias from :0 2024-04-03T16:26:27.9987071Z #133 Py_RunMain.localalias from :0 2024-04-03T16:26:27.9987244Z #134 Py_BytesMain from ??:0 2024-04-03T16:26:27.9987410Z #135 __libc_start_main from ??:0 2024-04-03T16:26:27.9987556Z #136 _start from ??:0 2024-04-03T16:26:27.9987696Z " 2024-04-03T16:26:27.9987717Z 2024-04-03T16:26:27.9988073Z To execute this test, run the following from the base repo dir: 2024-04-03T16:26:27.9988869Z python test/test_foreach.py -k test_binary_op_list_error_cases__foreach_add_cuda_float16 2024-04-03T16:26:27.9988891Z 2024-04-03T16:26:27.9989346Z This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0 2024-04-03T16:26:27.9989656Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-04-03T16:26:27.9990017Z ================== 1 failed, 116 deselected, 2 rerun in 1.51s ================== 2024-04-03T16:26:27.9990165Z Got exit code 1 2024-04-03T16:26:27.9990353Z Stopping at first consistent failure 2024-04-03T16:26:27.9991700Z The following tests failed and then succeeded when run in a new process['test/test_foreach.py::TestForeachCUDA::test_binary_op_float_inf_nan__foreach_add_cuda_float64'] 2024-04-03T16:26:27.9992746Z The following tests failed consistently: ['test/test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_float16'] 2024-04-03T16:26:27.9992768Z 2024-04-03T16:26:27.9993509Z FINISHED PRINTING LOG FILE of test_foreach 1/2 (test/test-reports/test_foreach_1.2_67e79d7959fa5e16_.log) 2024-04-03T16:26:27.9993555Z 2024-04-03T16:26:27.9993716Z test_foreach 1/2 failed! 2024-04-03T16:26:27.9993890Z Emiting td_test_failure_stats_v2 2024-04-03T16:26:28.7417663Z test_foreach 1/2 failed! 2024-04-03T16:26:29.0479720Z 2024-04-03T16:26:29.0481136Z real 1m28.988s 2024-04-03T16:26:29.0481845Z user 1m28.588s 2024-04-03T16:26:29.0482348Z sys 0m9.867s 2024-04-03T16:26:29.0534466Z ##[error]Process completed with exit code 1. 2024-04-03T16:26:29.0636837Z Prepare all required actions 2024-04-03T16:26:29.0637435Z Getting action download info 2024-04-03T16:26:29.2116035Z ##[group]Run ./.github/actions/pytest-cache-upload 2024-04-03T16:26:29.2116605Z with: 2024-04-03T16:26:29.2116958Z cache_dir: .pytest_cache 2024-04-03T16:26:29.2117386Z shard: 3 2024-04-03T16:26:29.2117790Z sha: d253bd7befc64137de913dbb8727b511f1052411 2024-04-03T16:26:29.2118358Z test_config: default 2024-04-03T16:26:29.2118890Z job_identifier: pull_linux-focal-cuda12.1-py3.10-gcc9 2024-04-03T16:26:29.2120101Z env: 2024-04-03T16:26:29.2120455Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:29.2121019Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:29.2121964Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:29.2122771Z ##[endgroup] 2024-04-03T16:26:29.2176453Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-04-03T16:26:29.2177190Z with: 2024-04-03T16:26:29.2177566Z shell: bash 2024-04-03T16:26:29.2177920Z timeout_minutes: 5 2024-04-03T16:26:29.2178316Z max_attempts: 5 2024-04-03T16:26:29.2178704Z retry_wait_seconds: 30 2024-04-03T16:26:29.2179246Z command: set -eu python3 -m pip install boto3==1.19.12 2024-04-03T16:26:29.2179889Z polling_interval_seconds: 1 2024-04-03T16:26:29.2180351Z warning_on_retry: true 2024-04-03T16:26:29.2180764Z continue_on_error: false 2024-04-03T16:26:29.2181174Z env: 2024-04-03T16:26:29.2181505Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:29.2182063Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:29.2183008Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:29.2183832Z ##[endgroup] 2024-04-03T16:26:29.5785468Z Defaulting to user installation because normal site-packages is not writeable 2024-04-03T16:26:30.9512097Z Collecting boto3==1.19.12 2024-04-03T16:26:30.9696572Z Downloading boto3-1.19.12-py3-none-any.whl (131 kB) 2024-04-03T16:26:31.0358887Z Collecting s3transfer<0.6.0,>=0.5.0 2024-04-03T16:26:31.0402954Z Downloading s3transfer-0.5.2-py3-none-any.whl (79 kB) 2024-04-03T16:26:31.0843567Z Collecting jmespath<1.0.0,>=0.7.1 2024-04-03T16:26:31.1153346Z Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB) 2024-04-03T16:26:32.7228068Z Collecting botocore<1.23.0,>=1.22.12 2024-04-03T16:26:32.7299438Z Downloading botocore-1.22.12-py3-none-any.whl (8.1 MB) 2024-04-03T16:26:32.9266645Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /home/ec2-user/.local/lib/python3.7/site-packages (from botocore<1.23.0,>=1.22.12->boto3==1.19.12) (1.26.18) 2024-04-03T16:26:32.9997989Z Collecting python-dateutil<3.0.0,>=2.1 2024-04-03T16:26:33.0052072Z Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-04-03T16:26:33.0654151Z Collecting six>=1.5 2024-04-03T16:26:33.0697753Z Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-03T16:26:33.1560545Z Installing collected packages: jmespath, six, python-dateutil, botocore, s3transfer, boto3 2024-04-03T16:26:34.0437399Z Successfully installed boto3-1.19.12 botocore-1.22.12 jmespath-0.10.0 python-dateutil-2.9.0.post0 s3transfer-0.5.2 six-1.16.0 2024-04-03T16:26:34.2962678Z Command completed after 1 attempt(s). 2024-04-03T16:26:34.3022580Z ##[group]Run python3 .github/scripts/pytest_cache.py \ 2024-04-03T16:26:34.3023309Z python3 .github/scripts/pytest_cache.py \ 2024-04-03T16:26:34.3023899Z  --upload \ 2024-04-03T16:26:34.3024432Z  --cache_dir $GITHUB_WORKSPACE/$CACHE_DIR \ 2024-04-03T16:26:34.3025050Z  --pr_identifier $GITHUB_REF \ 2024-04-03T16:26:34.3025643Z  --job_identifier $JOB_IDENTIFIER \ 2024-04-03T16:26:34.3026190Z  --sha $SHA \ 2024-04-03T16:26:34.3026643Z  --test_config $TEST_CONFIG \ 2024-04-03T16:26:34.3027148Z  --shard $SHARD \ 2024-04-03T16:26:34.3027595Z  --repo $REPO \ 2024-04-03T16:26:34.3028055Z  --temp_dir $RUNNER_TEMP \ 2024-04-03T16:26:34.3036825Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:26:34.3037477Z env: 2024-04-03T16:26:34.3037817Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:34.3038376Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:34.3039576Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:34.3040434Z CACHE_DIR: .pytest_cache 2024-04-03T16:26:34.3040993Z JOB_IDENTIFIER: pull_linux-focal-cuda12.1-py3.10-gcc9 2024-04-03T16:26:34.3041670Z SHA: d253bd7befc64137de913dbb8727b511f1052411 2024-04-03T16:26:34.3042229Z TEST_CONFIG: default 2024-04-03T16:26:34.3042621Z SHARD: 3 2024-04-03T16:26:34.3042951Z REPO: pytorch/pytorch 2024-04-03T16:26:34.3043361Z ##[endgroup] 2024-04-03T16:26:34.6891090Z PR identifier for `refs/pull/123154/merge` is `bc00947e1b58ab89c2c1cadfe46d41b7` 2024-04-03T16:26:34.6894848Z Uploading cache with args Namespace(bucket=None, cache_dir='/home/ec2-user/actions-runner/_work/pytorch/pytorch/.pytest_cache', download=False, job_identifier='pull_linux-focal-cuda12.1-py3.10-gcc9', pr_identifier='refs/pull/123154/merge', repo='pytorch/pytorch', sha='d253bd7befc64137de913dbb8727b511f1052411', shard='3', temp_dir='/home/ec2-user/actions-runner/_work/_temp', test_config='default', upload=True) 2024-04-03T16:26:34.6898084Z Zipping /home/ec2-user/actions-runner/_work/pytorch/pytorch/.pytest_cache 2024-04-03T16:26:34.6900260Z to /home/ec2-user/actions-runner/_work/_temp/zip-upload/pytest_cache/pytorch/pytorch/bc00947e1b58ab89c2c1cadfe46d41b7/pull_linux-focal-cuda12_1-py3_10-gcc9/d253bd7befc64137de913dbb8727b511f1052411/default/3 2024-04-03T16:26:34.6903468Z Uploading /home/ec2-user/actions-runner/_work/_temp/zip-upload/pytest_cache/pytorch/pytorch/bc00947e1b58ab89c2c1cadfe46d41b7/pull_linux-focal-cuda12_1-py3_10-gcc9/d253bd7befc64137de913dbb8727b511f1052411/default/3.zip 2024-04-03T16:26:34.6906457Z to s3://gha-artifacts/pytest_cache/pytorch/pytorch/bc00947e1b58ab89c2c1cadfe46d41b7/pull_linux-focal-cuda12_1-py3_10-gcc9/d253bd7befc64137de913dbb8727b511f1052411/default/3.zip 2024-04-03T16:26:34.7325406Z ##[group]Run cat test/**/*_toprint.log || true 2024-04-03T16:26:34.7326104Z cat test/**/*_toprint.log || true 2024-04-03T16:26:34.7334919Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:26:34.7335544Z env: 2024-04-03T16:26:34.7335867Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:34.7336566Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:34.7337574Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:34.7338379Z ##[endgroup] 2024-04-03T16:26:34.7415731Z Test results will be stored in test-reports/python-pytest/test_ops_jit/test_ops_jit-cd7f2ffcdca91c39.xml 2024-04-03T16:26:34.7417998Z ============================= test session starts ============================== 2024-04-03T16:26:34.7420242Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.4.0 -- /opt/conda/envs/py_3.10/bin/python 2024-04-03T16:26:34.7421023Z cachedir: .pytest_cache 2024-04-03T16:26:34.7422392Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-04-03T16:26:34.7423530Z rootdir: /var/lib/jenkins/workspace 2024-04-03T16:26:34.7424015Z configfile: pytest.ini 2024-04-03T16:26:34.7425007Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-04-03T16:26:34.7425992Z collecting ... collected 1107 items 2024-04-03T16:26:34.7426581Z stepcurrent: Cannot find last run test, not skipping 2024-04-03T16:26:34.7623759Z Running 367 items in this shard: test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_asin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_atanh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_cat_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_div_trunc_rounding_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_erf_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_erfc_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_expm1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_le_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_linalg_det_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_linalg_householder_product_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_linalg_matrix_power_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_log_softmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_logsumexp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_matmul_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_matrix_exp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_max_binary_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_mul_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_mvlgamma_mvlgamma_p_5_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_neg_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_nn_functional_conv1d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_nn_functional_group_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_nn_functional_layer_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_round_decimals_3_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_sigmoid_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_softmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_softmax_with_dtype_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_transpose_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_H_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_H_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_T_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___radd___cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rmatmul___cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rmatmul___cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rsub___cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__batch_norm_with_update_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__chunk_cat_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__chunk_cat_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__native_batch_norm_legit_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__segment_reduce_lengths_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__softmax_backward_data_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__upsample_bilinear2d_aa_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_add_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addbmm_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addcdiv_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addcmul_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addmv_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addr_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_all_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_allclose_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_amin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_aminmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_angle_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_any_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_argmin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_argwhere_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_scatter_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_asinh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atan_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atleast_3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bernoulli_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bfloat16_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bool_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_broadcast_tensors_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_broadcast_to_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_byte_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_byte_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ceil_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_inverse_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_inverse_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_solve_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_solve_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_chunk_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_chunk_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_clamp_max_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_clone_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_combinations_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_complex_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_conj_physical_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_conj_physical_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_constant_pad_nd_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_copysign_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_corrcoef_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cos_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_count_nonzero_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cov_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cummin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cumprod_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cumsum_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cumsum_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diag_embed_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diag_embed_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagonal_copy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagonal_scatter_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diff_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_digamma_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_dot_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_double_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_dsplit_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_empty_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_empty_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_empty_permuted_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_empty_strided_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_empty_strided_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_eq_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_equal_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_erf_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_expm1_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_fft2_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_fft_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_fftn_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_hfft2_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_hfftn_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_ifft_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_ifftshift_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_ihfft2_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_ihfft_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_irfft2_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_irfft_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_irfft_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fft_irfftn_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fill_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_flip_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_flip_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_flipud_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_floor_divide_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_fmin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_full_like_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_gather_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ge_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_geqrf_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_half_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_heaviside_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_hsplit_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_hstack_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_i0_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_add_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_fill_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_put_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_index_reduce_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_inner_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_int_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isfinite_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isinf_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isposinf_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_isreal_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_item_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_jiterator_2inputs_2outputs_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_jiterator_2inputs_2outputs_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_jiterator_binary_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_jiterator_binary_return_by_ref_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_kron_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ldexp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_lerp_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cholesky_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cholesky_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cholesky_ex_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cond_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_cross_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_det_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_det_singular_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_eigh_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_eigh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_inv_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_inv_ex_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lstsq_grad_oriented_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lu_factor_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_lu_factor_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_matrix_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_matrix_rank_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_matrix_rank_hermitian_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_matrix_rank_hermitian_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_multi_dot_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_norm_subgradients_at_zero_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_pinv_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_pinv_hermitian_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_qr_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_solve_triangular_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_solve_triangular_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_tensorsolve_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linalg_vector_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linspace_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_linspace_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_log10_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_log10_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_log1p_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_log_softmax_with_dtype_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logaddexp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logcumsumexp_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logdet_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logical_not_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logical_or_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logical_or_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_logspace_tensor_overload_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_lu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_lu_solve_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_lu_solve_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mH_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_fill_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_mean_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_normalize_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_prod_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_prod_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_scatter_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_scatter_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_softmax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_std_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_masked_std_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_matmul_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_max_reduction_no_dim_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_max_reduction_with_dim_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_maximum_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_meshgrid_variadic_tensors_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_meshgrid_variadic_tensors_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_min_binary_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_movedim_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_msort_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mul_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mv_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_mv_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nanmedian_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nansum_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_native_layer_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_neg_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_new_empty_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_new_full_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_new_zeros_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_adaptive_avg_pool1d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_adaptive_max_pool1d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_adaptive_max_pool2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_adaptive_max_pool3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_alpha_dropout_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_batch_norm_without_cudnn_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_celu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv1d_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_conv_transpose3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_cosine_embedding_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_cross_entropy_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_ctc_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_dropout2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_embedding_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_gaussian_nll_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_group_norm_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_hardsigmoid_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_linear_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_pool2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_pool3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_unpool1d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_unpool2d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_unpool3d_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_max_unpool3d_grad_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_mish_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_multilabel_margin_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_normalize_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pad_circular_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pad_reflect_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pad_replicate_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pad_replicate_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pdist_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pixel_unshuffle_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_pixel_unshuffle_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_relu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_rms_norm_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_selu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_softmin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_softmin_with_dtype_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_softsign_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_tanhshrink_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_tanhshrink_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_triplet_margin_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_triplet_margin_with_distance_loss_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_upsample_bilinear_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nn_functional_upsample_nearest_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_nonzero_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_norm_inf_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_normal_number_mean_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ones_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ones_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ormqr_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_permute_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_pinverse_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_polygamma_polygamma_n_1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_polygamma_polygamma_n_4_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_put_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_randint_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_randn_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_randn_like_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ravel_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_real_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_reciprocal_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_renorm_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_repeat_interleave_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_reshape_as_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_roll_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_rot90_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_round_decimals_neg_3_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_rsqrt_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_rsqrt_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_rsub_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scalar_tensor_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_reduce_amax_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_reduce_amin_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_reduce_mean_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_reduce_prod_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_scatter_reduce_sum_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_select_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_signal_windows_cosine_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_signbit_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sin_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sinc_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sinh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_slice_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_softmax_with_dtype_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_softmax_with_dtype_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_airy_ai_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_bessel_j0_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_bessel_y0_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_bessel_y1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_hermite_polynomial_he_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_i1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_polygamma_special_polygamma_n_0_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_scaled_modified_bessel_k1_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_shifted_chebyshev_polynomial_u_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_xlog1py_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_special_zeta_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_with_sizes_copy_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_split_with_sizes_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sqrt_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_square_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_std_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_std_mean_unbiased_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_std_unbiased_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_std_unbiased_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sub_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_sum_to_size_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_svd_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_svd_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_t_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_tanh_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_tensordot_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_transpose_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_trapezoid_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_trapz_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_triu_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unflatten_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unfold_copy_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unsafe_chunk_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unsafe_split_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unsafe_split_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unsqueeze_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_unsqueeze_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_var_mean_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_as_complex_cuda_float32, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_as_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_view_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_vstack_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_where_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_zeros_cuda_complex64, test/test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_zeros_cuda_float32 2024-04-03T16:26:34.7818515Z 2024-04-03T16:26:34.7819061Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_asin_cuda_float32 PASSED [1.0088s] [ 0%] 2024-04-03T16:26:34.7820289Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_atanh_cuda_float32 PASSED [0.0163s] [ 0%] 2024-04-03T16:26:34.7821490Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_cat_cuda_float32 XFAIL [0.0107s] [ 0%] 2024-04-03T16:26:34.7822774Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_div_trunc_rounding_cuda_float32 PASSED [0.0500s] [ 1%] 2024-04-03T16:26:34.7824036Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_erf_cuda_float32 PASSED [0.0132s] [ 1%] 2024-04-03T16:26:34.7825240Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_erfc_cuda_float32 PASSED [0.2118s] [ 1%] 2024-04-03T16:26:34.7826455Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_expm1_cuda_float32 PASSED [0.0356s] [ 1%] 2024-04-03T16:26:34.7827753Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_le_cuda_float32 PASSED [0.0227s] [ 2%] 2024-04-03T16:26:34.7828876Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_linalg_det_cuda_float32 PASSED [0.0720s] [ 2%] 2024-04-03T16:26:34.7830246Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_linalg_householder_product_cuda_float32 PASSED [0.1646s] [ 2%] 2024-04-03T16:26:34.7831656Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_linalg_matrix_power_cuda_float32 PASSED [0.0120s] [ 2%] 2024-04-03T16:26:34.7833117Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_log_softmax_cuda_float32 PASSED [0.0506s] [ 3%] 2024-04-03T16:26:34.7834420Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_logsumexp_cuda_float32 PASSED [0.3736s] [ 3%] 2024-04-03T16:26:34.7835666Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_matmul_cuda_float32 PASSED [0.0116s] [ 3%] 2024-04-03T16:26:34.7836889Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_matrix_exp_cuda_float32 PASSED [0.1019s] [ 4%] 2024-04-03T16:26:34.7838376Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_max_binary_cuda_float32 XFAIL [0.0371s] [ 4%] 2024-04-03T16:26:34.7839970Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_mul_cuda_float32 PASSED [0.0149s] [ 4%] 2024-04-03T16:26:34.7841286Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_mvlgamma_mvlgamma_p_5_cuda_float32 PASSED [0.3293s] [ 4%] 2024-04-03T16:26:34.7842558Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_neg_cuda_float32 PASSED [0.0150s] [ 5%] 2024-04-03T16:26:34.7843863Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_nn_functional_conv1d_cuda_float32 PASSED [0.1242s] [ 5%] 2024-04-03T16:26:34.7845281Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_nn_functional_group_norm_cuda_float32 PASSED [0.0435s] [ 5%] 2024-04-03T16:26:34.7846714Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_nn_functional_layer_norm_cuda_float32 PASSED [0.0135s] [ 5%] 2024-04-03T16:26:34.7848160Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_round_decimals_3_cuda_float32 SKIPPED [0.0003s] (Skipped!) [ 6%] 2024-04-03T16:26:34.7849519Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_sigmoid_cuda_float32 PASSED [0.0123s] [ 6%] 2024-04-03T16:26:34.7850754Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_softmax_cuda_float32 PASSED [0.0148s] [ 6%] 2024-04-03T16:26:34.7852032Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_softmax_with_dtype_cuda_float32 PASSED [0.0122s] [ 7%] 2024-04-03T16:26:34.7853348Z test_ops_jit.py::TestJitCUDA::test_jit_alias_remapping_transpose_cuda_float32 PASSED [0.0246s] [ 7%] 2024-04-03T16:26:34.7854601Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_H_cuda_complex64 XFAIL [0.0074s] [ 7%] 2024-04-03T16:26:34.7855822Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_H_cuda_float32 XFAIL [0.0053s] [ 7%] 2024-04-03T16:26:34.7857003Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_T_cuda_float32 XFAIL [0.0055s] [ 8%] 2024-04-03T16:26:34.7858268Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___radd___cuda_complex64 XFAIL [0.0120s] [ 8%] 2024-04-03T16:26:34.7859589Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rmatmul___cuda_complex64 XFAIL [0.0159s] [ 8%] 2024-04-03T16:26:34.7860890Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rmatmul___cuda_float32 XFAIL [0.0091s] [ 8%] 2024-04-03T16:26:34.7862188Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit___rsub___cuda_float32 XFAIL [0.0070s] [ 9%] 2024-04-03T16:26:34.7863564Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__batch_norm_with_update_cuda_float32 XFAIL [0.0268s] [ 9%] 2024-04-03T16:26:34.7864960Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__chunk_cat_cuda_complex64 PASSED [0.4864s] [ 9%] 2024-04-03T16:26:34.7866273Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__chunk_cat_cuda_float32 PASSED [0.3445s] [ 10%] 2024-04-03T16:26:34.7867665Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__native_batch_norm_legit_cuda_float32 XFAIL [0.2184s] [ 10%] 2024-04-03T16:26:34.7869211Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__segment_reduce_lengths_cuda_float32 SKIPPED [0.0003s] (Skipped!) [ 10%] 2024-04-03T16:26:34.7870743Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__softmax_backward_data_cuda_float32 XFAIL [0.0689s] [ 10%] 2024-04-03T16:26:34.7872173Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit__upsample_bilinear2d_aa_cuda_float32 XFAIL [0.0378s] [ 11%] 2024-04-03T16:26:34.7873680Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_add_cuda_complex64 PASSED [3.2348s] [ 11%] 2024-04-03T16:26:34.7874995Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addbmm_cuda_complex64 PASSED [3.6301s] [ 11%] 2024-04-03T16:26:34.7876337Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addcdiv_cuda_complex64 PASSED [5.4274s] [ 11%] 2024-04-03T16:26:34.7877622Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addcmul_cuda_complex64 PASSED [4.7251s] [ 12%] 2024-04-03T16:26:34.7879029Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addmv_cuda_float32 PASSED [0.8184s] [ 12%] 2024-04-03T16:26:34.7880698Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_addr_cuda_complex64 PASSED [1.1734s] [ 12%] 2024-04-03T16:26:34.7881954Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_all_cuda_complex64 PASSED [1.2373s] [ 13%] 2024-04-03T16:26:34.7883313Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_allclose_cuda_float32 SKIPPED [0.0003s] (Skipped!) [ 13%] 2024-04-03T16:26:34.7884668Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_amin_cuda_float32 PASSED [2.2750s] [ 13%] 2024-04-03T16:26:34.7885943Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_aminmax_cuda_float32 PASSED [0.2705s] [ 13%] 2024-04-03T16:26:34.7887194Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_angle_cuda_float32 PASSED [0.1057s] [ 14%] 2024-04-03T16:26:34.7888447Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_any_cuda_float32 PASSED [0.6332s] [ 14%] 2024-04-03T16:26:34.7889700Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_argmin_cuda_float32 PASSED [0.4132s] [ 14%] 2024-04-03T16:26:34.7890987Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_argwhere_cuda_complex64 PASSED [0.3548s] [ 14%] 2024-04-03T16:26:34.7892320Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_cuda_complex64 PASSED [1.2887s] [ 15%] 2024-04-03T16:26:34.7893699Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_as_strided_scatter_cuda_float32 PASSED [0.9171s] [ 15%] 2024-04-03T16:26:34.7895037Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_asinh_cuda_float32 PASSED [0.1150s] [ 15%] 2024-04-03T16:26:34.7896271Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atan_cuda_float32 PASSED [0.1167s] [ 16%] 2024-04-03T16:26:34.7897546Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_atleast_3d_cuda_float32 XFAIL [0.0759s] [ 16%] 2024-04-03T16:26:34.7898857Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bernoulli_cuda_float32 XFAIL [0.0052s] [ 16%] 2024-04-03T16:26:34.7900142Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bfloat16_cuda_float32 XFAIL [0.0060s] [ 16%] 2024-04-03T16:26:34.7901374Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_bool_cuda_float32 XFAIL [0.0056s] [ 17%] 2024-04-03T16:26:34.7902709Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_broadcast_tensors_cuda_complex64 PASSED [0.2000s] [ 17%] 2024-04-03T16:26:34.7904116Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_broadcast_to_cuda_complex64 PASSED [1.6388s] [ 17%] 2024-04-03T16:26:34.7905409Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_byte_cuda_complex64 XFAIL [0.0369s] [ 17%] 2024-04-03T16:26:34.7906653Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_byte_cuda_float32 XFAIL [0.0333s] [ 18%] 2024-04-03T16:26:34.7907893Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_ceil_cuda_float32 PASSED [0.1139s] [ 18%] 2024-04-03T16:26:34.7909195Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_cuda_complex64 PASSED [3.8646s] [ 18%] 2024-04-03T16:26:34.7910549Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_inverse_cuda_complex64 PASSED [3.0074s] [ 19%] 2024-04-03T16:26:34.7911999Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_inverse_cuda_float32 PASSED [1.4103s] [ 19%] 2024-04-03T16:26:34.7913522Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_solve_cuda_complex64 PASSED [3.1754s] [ 19%] 2024-04-03T16:26:34.7914916Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cholesky_solve_cuda_float32 PASSED [1.4996s] [ 19%] 2024-04-03T16:26:34.7916242Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_chunk_cuda_complex64 PASSED [1.0062s] [ 20%] 2024-04-03T16:26:34.7917521Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_chunk_cuda_float32 PASSED [0.4409s] [ 20%] 2024-04-03T16:26:34.7918941Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_clamp_max_cuda_float32 PASSED [0.9993s] [ 20%] 2024-04-03T16:26:34.7920651Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_clone_cuda_float32 PASSED [0.2083s] [ 20%] 2024-04-03T16:26:34.7921961Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_combinations_cuda_float32 PASSED [1.2561s] [ 21%] 2024-04-03T16:26:34.7923278Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_complex_cuda_float32 PASSED [0.5365s] [ 21%] 2024-04-03T16:26:34.7924609Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_conj_physical_cuda_complex64 PASSED [0.2220s] [ 21%] 2024-04-03T16:26:34.7926050Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_conj_physical_cuda_float32 SKIPPED [0.0003s] (Skipped!) [ 22%] 2024-04-03T16:26:34.7927484Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_constant_pad_nd_cuda_float32 PASSED [2.8549s] [ 22%] 2024-04-03T16:26:34.7928830Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_copysign_cuda_float32 PASSED [1.0180s] [ 22%] 2024-04-03T16:26:34.7930114Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_corrcoef_cuda_float32 PASSED [0.6278s] [ 22%] 2024-04-03T16:26:34.7931400Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cos_cuda_complex64 PASSED [1.6082s] [ 23%] 2024-04-03T16:26:34.7932707Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_count_nonzero_cuda_float32 PASSED [0.6442s] [ 23%] 2024-04-03T16:26:34.7934012Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cov_cuda_complex64 XFAIL [0.3386s] [ 23%] 2024-04-03T16:26:34.7935255Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cummin_cuda_float32 PASSED [0.3565s] [ 23%] 2024-04-03T16:26:34.7936541Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cumprod_cuda_float32 PASSED [2.2764s] [ 24%] 2024-04-03T16:26:34.7937833Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cumsum_cuda_complex64 PASSED [0.9278s] [ 24%] 2024-04-03T16:26:34.7939115Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_cumsum_cuda_float32 PASSED [0.4688s] [ 24%] 2024-04-03T16:26:34.7940424Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diag_embed_cuda_complex64 PASSED [3.5045s] [ 25%] 2024-04-03T16:26:34.7941758Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diag_embed_cuda_float32 PASSED [1.7120s] [ 25%] 2024-04-03T16:26:34.7943090Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagonal_copy_cuda_float32 PASSED [0.8780s] [ 25%] 2024-04-03T16:26:34.7944447Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diagonal_scatter_cuda_float32 PASSED [1.7788s] [ 25%] 2024-04-03T16:26:34.7945771Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_diff_cuda_float32 PASSED [8.1851s] [ 26%] 2024-04-03T16:26:34.7946911Z test_ops_jit.py::TestJitCUDA::test_variant_consistency_jit_digamma_cuda_float32 2024-04-03T16:26:34.7984412Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-04-03T16:26:34.7985052Z kill "$MONITOR_SCRIPT_PID" 2024-04-03T16:26:34.7992980Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:26:34.7993606Z env: 2024-04-03T16:26:34.7993925Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:34.7994484Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:34.7995423Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:34.7996254Z MONITOR_SCRIPT_PID: 128824 2024-04-03T16:26:34.7996692Z ##[endgroup] 2024-04-03T16:26:34.8212748Z Prepare all required actions 2024-04-03T16:26:34.8213369Z Getting action download info 2024-04-03T16:26:34.9405730Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-04-03T16:26:35.2555072Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-04-03T16:26:35.2555638Z with: 2024-04-03T16:26:35.2556205Z file-suffix: test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647 2024-04-03T16:26:35.2557135Z s3-bucket: gha-artifacts 2024-04-03T16:26:35.2557528Z env: 2024-04-03T16:26:35.2557854Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:35.2558408Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:35.2559581Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:35.2560415Z ##[endgroup] 2024-04-03T16:26:35.2593256Z ##[group]Run # Remove any previous test jsons if they exist 2024-04-03T16:26:35.2594094Z # Remove any previous test jsons if they exist 2024-04-03T16:26:35.2594715Z rm -f test-jsons-*.zip 2024-04-03T16:26:35.2595359Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-04-03T16:26:35.2604270Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:26:35.2604896Z env: 2024-04-03T16:26:35.2605211Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:35.2605767Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:35.2606707Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:35.2607778Z FILE_SUFFIX: test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647 2024-04-03T16:26:35.2608466Z ##[endgroup] 2024-04-03T16:26:35.2851139Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-04-03T16:26:35.2887392Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-04-03T16:26:35.2888278Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-04-03T16:26:35.2895795Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-04-03T16:26:35.2897046Z adding: test/.pytorch-slow-tests.json (deflated 59%) 2024-04-03T16:26:35.2905543Z adding: test/.pytorch-disabled-tests.json (deflated 84%) 2024-04-03T16:26:35.2938765Z ##[group]Run # Remove any previous test reports if they exist 2024-04-03T16:26:35.2939572Z # Remove any previous test reports if they exist 2024-04-03T16:26:35.2940222Z rm -f test-reports-*.zip 2024-04-03T16:26:35.2940950Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-04-03T16:26:35.2949028Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:26:35.2949629Z env: 2024-04-03T16:26:35.2949953Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:35.2950508Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:35.2951430Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:35.2952489Z FILE_SUFFIX: test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647 2024-04-03T16:26:35.2953200Z ##[endgroup] 2024-04-03T16:26:35.3171299Z adding: test/test-reports/python-pytest/test_foreach/test_foreach-9cbc8d00a0d5d7a6.xml (deflated 91%) 2024-04-03T16:26:35.3194876Z adding: test/test-reports/python-pytest/test_foreach/test_foreach-273a9009fcdeef67.xml (deflated 92%) 2024-04-03T16:26:35.3217988Z adding: test/test-reports/python-pytest/test_foreach/test_foreach-e74386081b5ae225.xml (deflated 92%) 2024-04-03T16:26:35.3240571Z adding: test/test-reports/python-pytest/test_foreach/test_foreach-26a998c1bb36d87a.xml (deflated 92%) 2024-04-03T16:26:35.3274378Z ##[group]Run # Remove any previous test reports if they exist 2024-04-03T16:26:35.3275196Z # Remove any previous test reports if they exist 2024-04-03T16:26:35.3275828Z rm -f logs-*.zip 2024-04-03T16:26:35.3276664Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-04-03T16:26:35.3277818Z # so check to see if the file exists first 2024-04-03T16:26:35.3278445Z if [ -f 'usage_log.txt' ]; then 2024-04-03T16:26:35.3279571Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-04-03T16:26:35.3280196Z fi 2024-04-03T16:26:35.3280462Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-04-03T16:26:35.3281178Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-04-03T16:26:35.3281787Z fi 2024-04-03T16:26:35.3289674Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:26:35.3290467Z env: 2024-04-03T16:26:35.3290803Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:35.3291350Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:35.3292297Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:35.3293350Z FILE_SUFFIX: test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647 2024-04-03T16:26:35.3294056Z ##[endgroup] 2024-04-03T16:26:35.3345339Z adding: usage_log.txt (deflated 95%) 2024-04-03T16:26:35.3573617Z adding: test/test-reports/test_ops_jit_2.3_deqwsqes_toprint.log (deflated 91%) 2024-04-03T16:26:35.3634203Z adding: test/test-reports/test_foreach_1.2_67e79d7959fa5e16_.log (deflated 94%) 2024-04-03T16:26:35.3702314Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-04-03T16:26:35.3702849Z with: 2024-04-03T16:26:35.3703184Z s3-bucket: gha-artifacts 2024-04-03T16:26:35.3703714Z s3-prefix: pytorch/pytorch/8536950919/2/artifact 2024-04-03T16:26:35.3704294Z retention-days: 14 2024-04-03T16:26:35.3704702Z if-no-files-found: warn 2024-04-03T16:26:35.3705141Z path: test-jsons-*.zip 2024-04-03T16:26:35.3705545Z name: artifact 2024-04-03T16:26:35.3705918Z region: us-east-1 2024-04-03T16:26:35.3706285Z env: 2024-04-03T16:26:35.3706599Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:35.3707160Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:35.3708116Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:35.3708931Z ##[endgroup] 2024-04-03T16:26:35.8188284Z NOTE: s3-prefix specified, ignoring name parameter 2024-04-03T16:26:35.8189733Z With the provided path, there will be 1 file uploaded 2024-04-03T16:26:35.8190635Z Uploading to s3 prefix: pytorch/pytorch/8536950919/2/artifact 2024-04-03T16:26:35.8237781Z Starting upload of test-jsons-test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647.zip 2024-04-03T16:26:35.9831971Z Finished upload of test-jsons-test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647.zip 2024-04-03T16:26:36.0018189Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-04-03T16:26:36.0018722Z with: 2024-04-03T16:26:36.0019063Z s3-bucket: gha-artifacts 2024-04-03T16:26:36.0019587Z s3-prefix: pytorch/pytorch/8536950919/2/artifact 2024-04-03T16:26:36.0020154Z retention-days: 14 2024-04-03T16:26:36.0020568Z if-no-files-found: error 2024-04-03T16:26:36.0021014Z path: test-reports-*.zip 2024-04-03T16:26:36.0021421Z name: artifact 2024-04-03T16:26:36.0021811Z region: us-east-1 2024-04-03T16:26:36.0022177Z env: 2024-04-03T16:26:36.0022488Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:36.0023042Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:36.0023974Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:36.0024778Z ##[endgroup] 2024-04-03T16:26:36.4517974Z NOTE: s3-prefix specified, ignoring name parameter 2024-04-03T16:26:36.4519892Z With the provided path, there will be 1 file uploaded 2024-04-03T16:26:36.4520725Z Uploading to s3 prefix: pytorch/pytorch/8536950919/2/artifact 2024-04-03T16:26:36.4567359Z Starting upload of test-reports-test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647.zip 2024-04-03T16:26:36.7024365Z Finished upload of test-reports-test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647.zip 2024-04-03T16:26:36.7199028Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-04-03T16:26:36.7200242Z with: 2024-04-03T16:26:36.7200600Z s3-bucket: gha-artifacts 2024-04-03T16:26:36.7201361Z s3-prefix: pytorch/pytorch/8536950919/2/artifact 2024-04-03T16:26:36.7201959Z retention-days: 14 2024-04-03T16:26:36.7202379Z if-no-files-found: ignore 2024-04-03T16:26:36.7202818Z path: logs-*.zip 2024-04-03T16:26:36.7203178Z name: artifact 2024-04-03T16:26:36.7203550Z region: us-east-1 2024-04-03T16:26:36.7203909Z env: 2024-04-03T16:26:36.7204223Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:36.7204785Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:36.7205904Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:36.7206713Z ##[endgroup] 2024-04-03T16:26:37.1774313Z NOTE: s3-prefix specified, ignoring name parameter 2024-04-03T16:26:37.1775760Z With the provided path, there will be 1 file uploaded 2024-04-03T16:26:37.1776433Z Uploading to s3 prefix: pytorch/pytorch/8536950919/2/artifact 2024-04-03T16:26:37.1824783Z Starting upload of logs-test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647.zip 2024-04-03T16:26:37.4042837Z Finished upload of logs-test-default-3-5-linux.4xlarge.nvidia.gpu_23402695647.zip 2024-04-03T16:26:37.4237289Z ##[group]Run # shellcheck disable=SC2156 2024-04-03T16:26:37.4237966Z # shellcheck disable=SC2156 2024-04-03T16:26:37.4239300Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-04-03T16:26:37.4248378Z shell: /usr/bin/bash -e {0} 2024-04-03T16:26:37.4248805Z env: 2024-04-03T16:26:37.4249159Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:37.4249723Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:37.4250650Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:37.4251478Z ##[endgroup] 2024-04-03T16:26:37.6954028Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-04-03T16:26:37.6954665Z with: 2024-04-03T16:26:37.6955140Z name: coredumps-default-3-5-linux.4xlarge.nvidia.gpu 2024-04-03T16:26:37.6955776Z retention-days: 14 2024-04-03T16:26:37.6956174Z if-no-files-found: ignore 2024-04-03T16:26:37.6956618Z path: ./**/core.[1-9]* 2024-04-03T16:26:37.6957044Z s3-bucket: gha-artifacts 2024-04-03T16:26:37.6957458Z region: us-east-1 2024-04-03T16:26:37.6957825Z env: 2024-04-03T16:26:37.6958155Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:37.6958695Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:37.6959903Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:37.6960750Z ##[endgroup] 2024-04-03T16:26:48.5795622Z No files were found with the provided path: ./**/core.[1-9]*. No artifacts will be uploaded. 2024-04-03T16:26:48.6066590Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-04-03T16:26:48.6067314Z with: 2024-04-03T16:26:48.6067622Z env: 2024-04-03T16:26:48.6067957Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:48.6068514Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:48.6069459Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:48.6070277Z ##[endgroup] 2024-04-03T16:26:48.6095316Z ##[group]Run set -eou pipefail 2024-04-03T16:26:48.6095823Z set -eou pipefail 2024-04-03T16:26:48.6096259Z  2024-04-03T16:26:48.6096896Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-04-03T16:26:48.6097681Z for _ in $(seq 1440); do 2024-04-03T16:26:48.6098266Z  # Break if no ssh session exists anymore 2024-04-03T16:26:48.6098904Z  if [ "$(who)" = "" ]; then 2024-04-03T16:26:48.6099394Z  break 2024-04-03T16:26:48.6099837Z  fi 2024-04-03T16:26:48.6100190Z  echo "." 2024-04-03T16:26:48.6100590Z  sleep 5 2024-04-03T16:26:48.6100971Z done 2024-04-03T16:26:48.6109327Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:26:48.6109933Z env: 2024-04-03T16:26:48.6110267Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:48.6110830Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:48.6111752Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:48.6112642Z ##[endgroup] 2024-04-03T16:26:48.6138881Z Holding runner for 2 hours until all ssh sessions have logged out 2024-04-03T16:26:48.6191808Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-04-03T16:26:48.6192779Z # ignore expansion of "docker ps -q" since it could be empty 2024-04-03T16:26:48.6193720Z # shellcheck disable=SC2046 2024-04-03T16:26:48.6194306Z docker stop $(docker ps -q) || true 2024-04-03T16:26:48.6194890Z # Prune all of the docker images 2024-04-03T16:26:48.6195451Z docker system prune -af 2024-04-03T16:26:48.6204070Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:26:48.6204703Z env: 2024-04-03T16:26:48.6205028Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:48.6205593Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:48.6206537Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:48.6207348Z ##[endgroup] 2024-04-03T16:26:49.0921186Z 5f8b51bc22a6 2024-04-03T16:26:49.6754764Z Deleted Containers: 2024-04-03T16:26:49.6755932Z 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:49.6756511Z 2024-04-03T16:26:56.4077773Z Deleted Images: 2024-04-03T16:26:56.4080017Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9:16884143090f148aa56809ceba26eeb699b5dc49 2024-04-03T16:26:56.4083886Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn8-py3-gcc9@sha256:a560526b69e9ece9c90f2a23812ba2f1a57591b7cd437a945ebbb84c42e75301 2024-04-03T16:26:56.4086478Z deleted: sha256:4f8185cfd17efcb172142a7ee4ddc70e9bf6e340c2b0e42aa0def9ad36662df7 2024-04-03T16:26:56.4087874Z deleted: sha256:4a552cbb3870d73f1fa21a7f82984abc7730c23130cce95fcd75631cbf836fd2 2024-04-03T16:26:56.4089842Z deleted: sha256:61ba5cf5524d4451796031cf4c34968094da52e0db84642a723ace48d799d040 2024-04-03T16:26:56.4091578Z deleted: sha256:a945771f62bacc032282ece5c52150a7b66e1838f07e3941ee41adf00e45f975 2024-04-03T16:26:56.4093496Z deleted: sha256:e24b07fa010aa297f36e7af145e596b86c5b5a8e1574ef2ebf5a8965167a1486 2024-04-03T16:26:56.4094971Z deleted: sha256:565fdae7f0a54bc846c035ff44ef6fa14057d898ef6ca5b3d07e4328d126eb38 2024-04-03T16:26:56.4096030Z deleted: sha256:732b5ba916568738e2831410fb9688e404fbb7c70f8ec96a1fa77c1a4d10d168 2024-04-03T16:26:56.4097366Z deleted: sha256:9188fcddb13cd6ef435603efe6a381ada85e7c1e9fa23420b7fdea413c00630d 2024-04-03T16:26:56.4098475Z deleted: sha256:45ec4f15fd6913eb55e41869c157f5b423ee892c9e164d10d91dc168ce591001 2024-04-03T16:26:56.4099549Z deleted: sha256:30b52bca1e234ed4abffb25ccae8b785ce1c925b72e1bff15541d3b85c636f00 2024-04-03T16:26:56.4100616Z deleted: sha256:2405b1569dd9ec0ea29e28704bb9b19e14ca851bf0848a1f6c0ebaa158f00c06 2024-04-03T16:26:56.4101676Z deleted: sha256:cd308412d15f75d0b74a7058801f1a2e4ff294f3d72984a15129362f98fd0990 2024-04-03T16:26:56.4102714Z deleted: sha256:01921506bb7614f12860b5358ad69f47d5ec2aa6c0e038c5b0745a8cd5a0b02d 2024-04-03T16:26:56.4103733Z deleted: sha256:0b921e4ef98a571df357bb69129be7061330926b342b910b765f50474362ac51 2024-04-03T16:26:56.4104810Z deleted: sha256:f4baf78a972f17bd0a6a5aaa18e0a4228dbbbaf9f2fca052cf852ce5a97fc7be 2024-04-03T16:26:56.4105905Z deleted: sha256:2219f52ec962afe8b7555a65fdfa38ac2a226e0b112f67d5b6777e75ec610340 2024-04-03T16:26:56.4106944Z deleted: sha256:07144a192e3da9bb81926748081dc01278264f2c96dd9c2d7ac1a0a2ef49d999 2024-04-03T16:26:56.4107997Z deleted: sha256:bbb7b459981f391fac1f38fefa7a081045214f93d221da9304e065c79704c1da 2024-04-03T16:26:56.4109062Z deleted: sha256:aa0daa5259391defef9871cda8b538719363a75b0b5f5053f0df9038b61c8828 2024-04-03T16:26:56.4110128Z deleted: sha256:d4057f54a136eb6a8ccf5b852a0719b6906396f9d01b26e83ac498ac0681ba13 2024-04-03T16:26:56.4112010Z deleted: sha256:892b47bfa5eb57f64fd092309c104b9f88966953db82e712d722f9fbdfebe74a 2024-04-03T16:26:56.4113557Z deleted: sha256:c34b6960f8529b13356337b0b750f86a3d7d33f146bb3a6a1235c6c79e9a016b 2024-04-03T16:26:56.4114594Z deleted: sha256:3ef70d5dc46853dc0e7399174d7002b46b0291c9dd83431d5870c1361b4c99d0 2024-04-03T16:26:56.4115636Z deleted: sha256:ca58e07f9539956be424eab4038d457e5eacd4675bea293e802e153cc983e6ce 2024-04-03T16:26:56.4116691Z deleted: sha256:48c0599897703e03a6f2af11bf950a80651e5c4db4d749207db39a09ae3cc533 2024-04-03T16:26:56.4117958Z deleted: sha256:a3fb212f169569d18ccfb7dad2b5d5f2673a71b2f1538f37b0c607365806f306 2024-04-03T16:26:56.4118999Z deleted: sha256:27179020897580530f4f23992eb073476dbf1652f2f332d8f919f56ff6ee1654 2024-04-03T16:26:56.4120512Z deleted: sha256:9ccadefb09a21df35eaadaff98d71d6e98737e6d6344ae287c316062e1d99b14 2024-04-03T16:26:56.4121600Z deleted: sha256:440572f7ae8c7f1b504ca929c5116bfd3123edb56be5946a79272a48187dd5f8 2024-04-03T16:26:56.4122652Z deleted: sha256:d9b841b156a02212a50db1b61379862a21f39e9dcabb23c70660ecff0c6cca5b 2024-04-03T16:26:56.4123710Z deleted: sha256:456cae9b6cf91a64d98dd0ecef5230eccd0f7c8aea9d187f4e6d69dfdaadbb0a 2024-04-03T16:26:56.4124792Z deleted: sha256:24928fff1f2e15e4c13d55cb75150d8756e8f9a46b4603f7cb3a4a99e4c20b51 2024-04-03T16:26:56.4125828Z deleted: sha256:759516864394e4fd381128e4ea6528b7c39a11ca4b1405c9d48399aaf3966416 2024-04-03T16:26:56.4126862Z deleted: sha256:b0feec4a66379758e16787cfae0f9e039c15fbfdc0b9f3bc2b76f8372f7f5964 2024-04-03T16:26:56.4127941Z deleted: sha256:5aa94b7477b7188fcc29383d80c9550b3ca26e9fef19582c5916d7f2b8c968dc 2024-04-03T16:26:56.4128989Z deleted: sha256:45020225cfceb2d5405b59e2b17912228588d36d5b39b5d29001e0d850d0d26c 2024-04-03T16:26:56.4130039Z deleted: sha256:58ace2b1942d48bd6a3872cec58fd86d8464359de68c762d72672822d5feefe9 2024-04-03T16:26:56.4131092Z deleted: sha256:cb3914d3b0db92a2e11fdd0778a1a263ac96ab283b899b5cc313c1b306549a8e 2024-04-03T16:26:56.4132170Z deleted: sha256:0e559fb4d85fa5b89d02c9a2b8496200ad1bdf99e73dcb0202669eca1865924a 2024-04-03T16:26:56.4133256Z deleted: sha256:b8eabd831eb26fe4991ea6c04db5c1bc38d2b88220c8ba5dfd543d9727b78526 2024-04-03T16:26:56.4134358Z deleted: sha256:fbea950698003b0c93f1d80058c7f1f2b8a1c7730c6d6c8b678967cc90b7e2e9 2024-04-03T16:26:56.4135393Z deleted: sha256:5387c0b2c00411f7bb84aadbed7e02e1e911f2517673ec1660e692dc8867f488 2024-04-03T16:26:56.4136442Z deleted: sha256:2e83a68a0e0c6526545d7f3c3ba97a2727e618130ff665d7bcb29fcc39775033 2024-04-03T16:26:56.4137508Z deleted: sha256:f746ca426a86c645fef471d0edf40641ede95c2196a1c5879b71fd3a64c72872 2024-04-03T16:26:56.4138699Z deleted: sha256:7c21595c43019cc12188060d4e1da1905495cbb1099388e8aef9c70f12ef2a2c 2024-04-03T16:26:56.4139784Z deleted: sha256:b9a5611b90edcd367de9eef1e9faf1d1e404e40694a41f6ff0edf3b784348fe5 2024-04-03T16:26:56.4140857Z deleted: sha256:6ec4533a5e60d0435c6285f2448b23ab46d29aef85c1c2df9262457de13bdbaa 2024-04-03T16:26:56.4141887Z deleted: sha256:c9034f66b97a71bb375c0bc1f9c7566559ed1da355b595166dc985e4aa7c490e 2024-04-03T16:26:56.4142925Z deleted: sha256:56a169f01ddc877822f681b4661415125eecf669d971c142534d88644e0146ac 2024-04-03T16:26:56.4143964Z deleted: sha256:5df18b52e0667d1015e3287e9a9aaf4ff6987ff0c13911c04be999642739a4e5 2024-04-03T16:26:56.4145018Z deleted: sha256:8d21e461f8b5892b95caf26bbc59b653e9463371d724c78e6ec36239ed9fffe8 2024-04-03T16:26:56.4146044Z deleted: sha256:1f76d2478aa78a4883044823b5fefd003141f8ea74d5ccebad792b4514e6023e 2024-04-03T16:26:56.4147115Z deleted: sha256:16cda24fc887b760cd5f65f6fc7580f00560d155d8e2faddaf6b6991329f64ee 2024-04-03T16:26:56.4148193Z deleted: sha256:fef603634d48bf8d8e9ebb13856bd6cae40df3010854e9c0ab0fe51544427532 2024-04-03T16:26:56.4149239Z deleted: sha256:7822ddc6e8648d4c10d4ffc1bf05132f1304be3141a04be427fc9c818183d111 2024-04-03T16:26:56.4150312Z deleted: sha256:16d23f55f5c2ebdd0e85d1c7875bbc812e7b9a4b341c317f4a44f1a8df838d17 2024-04-03T16:26:56.4151364Z deleted: sha256:4621915c79495397e147f640c677cadfbb2873068eda0b1368ceb8b3a2d1e644 2024-04-03T16:26:56.4152389Z deleted: sha256:33bd0e5c04f34d20250b42e1ed3ae506a06b7aca100d8568ad6c0463f67d19fc 2024-04-03T16:26:56.4153463Z deleted: sha256:dafa1db3322c29ecb96f043da215dce6684543294fb35ae2e7af9096971b9172 2024-04-03T16:26:56.4154533Z deleted: sha256:eee1885bef1a4a99428312c94614215a1b46804f1debdaa7f3c8a6f7d1bd12b8 2024-04-03T16:26:56.4155595Z deleted: sha256:83313222739c055fae1808a61cdedfcbdd760cbe25e9b161fad15a3f88fc7ff5 2024-04-03T16:26:56.4156652Z deleted: sha256:c6a425a39598b012eee298bdbd7ed44dfcea231a5c0e4dd17023e7f1f030a2c4 2024-04-03T16:26:56.4157865Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-04-03T16:26:56.4158924Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-04-03T16:26:56.4160755Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-04-03T16:26:56.4161815Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-04-03T16:26:56.4162879Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-04-03T16:26:56.4163917Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-04-03T16:26:56.4165044Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-04-03T16:26:56.4166071Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-04-03T16:26:56.4167156Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-04-03T16:26:56.4168228Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-04-03T16:26:56.4169288Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-04-03T16:26:56.4169928Z 2024-04-03T16:26:56.4170120Z Total reclaimed space: 24.66GB 2024-04-03T16:26:56.4216567Z ##[group]Run set +e 2024-04-03T16:26:56.4217119Z set +e 2024-04-03T16:26:56.4217464Z set -x 2024-04-03T16:26:56.4217825Z  2024-04-03T16:26:56.4218164Z nvidia-smi 2024-04-03T16:26:56.4218931Z # NB: Surprisingly, nvidia-smi command returns successfully with return code 0 even in 2024-04-03T16:26:56.4220178Z # the case where the driver has already crashed as it still can get the driver version 2024-04-03T16:26:56.4221404Z # and some basic information like the bus ID. However, the rest of the information 2024-04-03T16:26:56.4222333Z # would be missing (ERR!), for example: 2024-04-03T16:26:56.4222882Z # 2024-04-03T16:26:56.4223400Z # +-----------------------------------------------------------------------------+ 2024-04-03T16:26:56.4224366Z # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | 2024-04-03T16:26:56.4225364Z # |-------------------------------+----------------------+----------------------+ 2024-04-03T16:26:56.4226299Z # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-04-03T16:26:56.4227387Z # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | 2024-04-03T16:26:56.4228316Z # | | | MIG M. | 2024-04-03T16:26:56.4229058Z # |===============================+======================+======================| 2024-04-03T16:26:56.4229851Z # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | 2024-04-03T16:26:56.4230812Z # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | 2024-04-03T16:26:56.4231693Z # | | | ERR! | 2024-04-03T16:26:56.4232480Z # +-------------------------------+----------------------+----------------------+ 2024-04-03T16:26:56.4233104Z # 2024-04-03T16:26:56.4233614Z # +-----------------------------------------------------------------------------+ 2024-04-03T16:26:56.4234448Z # | Processes: | 2024-04-03T16:26:56.4235355Z # | GPU GI CI PID Type Process name GPU Memory | 2024-04-03T16:26:56.4236253Z # | ID ID Usage | 2024-04-03T16:26:56.4237016Z # |=============================================================================| 2024-04-03T16:26:56.4237978Z # +-----------------------------------------------------------------------------+ 2024-04-03T16:26:56.4238605Z # 2024-04-03T16:26:56.4239782Z # This should be reported as a failure instead as it will guarantee to fail when 2024-04-03T16:26:56.4241037Z # Docker tries to run with --gpus all 2024-04-03T16:26:56.4241568Z # 2024-04-03T16:26:56.4242230Z # So, the correct check here is to query one of the missing piece of info like 2024-04-03T16:26:56.4243196Z # GPU name, so that the command can fail accordingly 2024-04-03T16:26:56.4244060Z nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-04-03T16:26:56.4244763Z NVIDIA_SMI_STATUS=$? 2024-04-03T16:26:56.4245214Z  2024-04-03T16:26:56.4245979Z # These are acceptable return code from nvidia-smi as copied from setup-nvidia GitHub action 2024-04-03T16:26:56.4247119Z if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then 2024-04-03T16:26:56.4248171Z  echo "NVIDIA driver installation has failed, shutting down the runner..." 2024-04-03T16:26:56.4249068Z  .github/scripts/stop_runner_service.sh 2024-04-03T16:26:56.4249617Z fi 2024-04-03T16:26:56.4249956Z  2024-04-03T16:26:56.4250838Z # For runner with multiple GPUs, we also want to confirm that the number of GPUs are the 2024-04-03T16:26:56.4252081Z # power of 2, i.e. 1, 2, 4, or 8. This is to avoid flaky test issue when one GPU fails 2024-04-03T16:26:56.4253069Z # https://github.com/pytorch/test-infra/issues/4000 2024-04-03T16:26:56.4253827Z GPU_COUNT=$(nvidia-smi --list-gpus | wc -l) 2024-04-03T16:26:56.4254426Z NVIDIA_SMI_STATUS=$? 2024-04-03T16:26:56.4254858Z  2024-04-03T16:26:56.4255620Z # These are acceptable return code from nvidia-smi as copied from setup-nvidia GitHub action 2024-04-03T16:26:56.4256781Z if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then 2024-04-03T16:26:56.4257814Z  echo "NVIDIA driver installation has failed, shutting down the runner..." 2024-04-03T16:26:56.4258710Z  .github/scripts/stop_runner_service.sh 2024-04-03T16:26:56.4259275Z fi 2024-04-03T16:26:56.4259620Z  2024-04-03T16:26:56.4260036Z # Check the GPU count to be a power of 2 2024-04-03T16:26:56.4261093Z if [ "$GPU_COUNT" -le 8 ] && [ "$GPU_COUNT" -ne 1 ] && [ "$GPU_COUNT" -ne 2 ] && [ "$GPU_COUNT" -ne 4 ] && [ "$GPU_COUNT" -ne 8 ]; then 2024-04-03T16:26:56.4262489Z  echo "NVIDIA driver detects $GPU_COUNT GPUs. The runner has a broken GPU, shutting it down..." 2024-04-03T16:26:56.4263486Z  .github/scripts/stop_runner_service.sh 2024-04-03T16:26:56.4264051Z fi 2024-04-03T16:26:56.4272363Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-04-03T16:26:56.4272988Z env: 2024-04-03T16:26:56.4273307Z GIT_DEFAULT_BRANCH: main 2024-04-03T16:26:56.4273879Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-04-03T16:26:56.4274815Z DOCKER_CONTAINER_ID: 5f8b51bc22a6e74aed8f4bcf4723b5972818aa509d04a8d0c14eb5b288d2b608 2024-04-03T16:26:56.4275790Z RUNNER_WORKSPACE: /home/ec2-user/actions-runner/_work/pytorch 2024-04-03T16:26:56.4276444Z ##[endgroup] 2024-04-03T16:26:56.4303409Z + nvidia-smi 2024-04-03T16:26:56.9980142Z Wed Apr 3 16:26:56 2024 2024-04-03T16:26:56.9980971Z +---------------------------------------------------------------------------------------+ 2024-04-03T16:26:56.9982003Z | NVIDIA-SMI 535.54.03 Driver Version: 535.54.03 CUDA Version: 12.2 | 2024-04-03T16:26:56.9982989Z |-----------------------------------------+----------------------+----------------------+ 2024-04-03T16:26:56.9983982Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-04-03T16:26:56.9985073Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-04-03T16:26:56.9986194Z | | | MIG M. | 2024-04-03T16:26:56.9986872Z |=========================================+======================+======================| 2024-04-03T16:26:57.0149803Z | 0 Tesla M60 Off | 00000000:00:1E.0 Off | 0 | 2024-04-03T16:26:57.0150712Z | N/A 29C P0 38W / 150W | 0MiB / 7680MiB | 1% Default | 2024-04-03T16:26:57.0151531Z | | | N/A | 2024-04-03T16:26:57.0152410Z +-----------------------------------------+----------------------+----------------------+ 2024-04-03T16:26:57.0153165Z 2024-04-03T16:26:57.0154018Z +---------------------------------------------------------------------------------------+ 2024-04-03T16:26:57.0154823Z | Processes: | 2024-04-03T16:26:57.0155717Z | GPU GI CI PID Type Process name GPU Memory | 2024-04-03T16:26:57.0156607Z | ID ID Usage | 2024-04-03T16:26:57.0157644Z |=======================================================================================| 2024-04-03T16:26:57.0158551Z | No running processes found | 2024-04-03T16:26:57.0159821Z +---------------------------------------------------------------------------------------+ 2024-04-03T16:26:57.2928053Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-04-03T16:26:57.8390097Z Tesla M60 2024-04-03T16:26:58.0889691Z + NVIDIA_SMI_STATUS=0 2024-04-03T16:26:58.0890328Z + '[' 0 -ne 0 ']' 2024-04-03T16:26:58.0895896Z ++ nvidia-smi --list-gpus 2024-04-03T16:26:58.0896596Z ++ wc -l 2024-04-03T16:26:58.9024347Z + GPU_COUNT=1 2024-04-03T16:26:58.9025079Z + NVIDIA_SMI_STATUS=0 2024-04-03T16:26:58.9025621Z + '[' 0 -ne 0 ']' 2024-04-03T16:26:58.9026237Z + '[' 1 -le 8 ']' 2024-04-03T16:26:58.9026971Z + '[' 1 -ne 1 ']' 2024-04-03T16:26:58.9102437Z Post job cleanup. 2024-04-03T16:26:58.9168391Z Post job cleanup. 2024-04-03T16:26:59.0348548Z [command]/usr/bin/git version 2024-04-03T16:26:59.0404131Z git version 2.40.1 2024-04-03T16:26:59.0467507Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/ac096764-bf62-492e-9fb4-f7bcb4d79398' before making global git config changes 2024-04-03T16:26:59.0469112Z Adding repository directory to the temporary git global config as a safe directory 2024-04-03T16:26:59.0475082Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-04-03T16:26:59.0517451Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-04-03T16:26:59.0554164Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-04-03T16:26:59.0886579Z Entering 'android/libs/fbjni' 2024-04-03T16:26:59.0939818Z Entering 'third_party/FP16' 2024-04-03T16:26:59.0992529Z Entering 'third_party/FXdiv' 2024-04-03T16:26:59.1046100Z Entering 'third_party/NNPACK' 2024-04-03T16:26:59.1101173Z Entering 'third_party/QNNPACK' 2024-04-03T16:26:59.1154741Z Entering 'third_party/VulkanMemoryAllocator' 2024-04-03T16:26:59.1207301Z Entering 'third_party/XNNPACK' 2024-04-03T16:26:59.1278946Z Entering 'third_party/benchmark' 2024-04-03T16:26:59.1331080Z Entering 'third_party/cpuinfo' 2024-04-03T16:26:59.1384814Z Entering 'third_party/cub' 2024-04-03T16:26:59.1436328Z Entering 'third_party/cudnn_frontend' 2024-04-03T16:26:59.1490846Z Entering 'third_party/cutlass' 2024-04-03T16:26:59.1551190Z Entering 'third_party/eigen' 2024-04-03T16:26:59.1608483Z Entering 'third_party/fbgemm' 2024-04-03T16:26:59.1659378Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-04-03T16:26:59.1710840Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-04-03T16:26:59.1762763Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-04-03T16:26:59.1819057Z Entering 'third_party/fbgemm/third_party/googletest' 2024-04-03T16:26:59.1869646Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-04-03T16:26:59.1921532Z Entering 'third_party/flatbuffers' 2024-04-03T16:26:59.1975999Z Entering 'third_party/fmt' 2024-04-03T16:26:59.2029577Z Entering 'third_party/foxi' 2024-04-03T16:26:59.2082641Z Entering 'third_party/gemmlowp/gemmlowp' 2024-04-03T16:26:59.2134575Z Entering 'third_party/gloo' 2024-04-03T16:26:59.2187492Z Entering 'third_party/googletest' 2024-04-03T16:26:59.2240827Z Entering 'third_party/ideep' 2024-04-03T16:26:59.2291682Z Entering 'third_party/ideep/mkl-dnn' 2024-04-03T16:26:59.2351014Z Entering 'third_party/ios-cmake' 2024-04-03T16:26:59.2404934Z Entering 'third_party/ittapi' 2024-04-03T16:26:59.2455526Z Entering 'third_party/kineto' 2024-04-03T16:26:59.2508116Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-04-03T16:26:59.2561449Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-04-03T16:26:59.2613287Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-04-03T16:26:59.2663841Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-04-03T16:26:59.2714994Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-04-03T16:26:59.2764564Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-04-03T16:26:59.2816653Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-04-03T16:26:59.2867669Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-04-03T16:26:59.2918710Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-04-03T16:26:59.2971479Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-04-03T16:26:59.3023513Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-04-03T16:26:59.3074627Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-04-03T16:26:59.3127609Z Entering 'third_party/mimalloc' 2024-04-03T16:26:59.3180084Z Entering 'third_party/nccl/nccl' 2024-04-03T16:26:59.3232251Z Entering 'third_party/neon2sse' 2024-04-03T16:26:59.3285739Z Entering 'third_party/nlohmann' 2024-04-03T16:26:59.3339800Z Entering 'third_party/onnx' 2024-04-03T16:26:59.3409912Z Entering 'third_party/onnx/third_party/benchmark' 2024-04-03T16:26:59.3462514Z Entering 'third_party/onnx/third_party/pybind11' 2024-04-03T16:26:59.3516243Z Entering 'third_party/onnx-tensorrt' 2024-04-03T16:26:59.3567836Z Entering 'third_party/onnx-tensorrt/third_party/onnx' 2024-04-03T16:26:59.3622811Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark' 2024-04-03T16:26:59.3674973Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11' 2024-04-03T16:26:59.3726501Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang' 2024-04-03T16:26:59.3783279Z Entering 'third_party/pocketfft' 2024-04-03T16:26:59.3835510Z Entering 'third_party/protobuf' 2024-04-03T16:26:59.3891493Z Entering 'third_party/protobuf/third_party/benchmark' 2024-04-03T16:26:59.3942777Z Entering 'third_party/protobuf/third_party/googletest' 2024-04-03T16:26:59.3996076Z Entering 'third_party/psimd' 2024-04-03T16:26:59.4048707Z Entering 'third_party/pthreadpool' 2024-04-03T16:26:59.4100891Z Entering 'third_party/pybind11' 2024-04-03T16:26:59.4153059Z Entering 'third_party/python-peachpy' 2024-04-03T16:26:59.4206547Z Entering 'third_party/sleef' 2024-04-03T16:26:59.4258233Z Entering 'third_party/tbb' 2024-04-03T16:26:59.4312503Z Entering 'third_party/tensorpipe' 2024-04-03T16:26:59.4364201Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-04-03T16:26:59.4414702Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-04-03T16:26:59.4489013Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-04-03T16:26:59.4520507Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-04-03T16:26:59.4569744Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-04-03T16:26:59.4623760Z Entering 'third_party/zstd' 2024-04-03T16:26:59.4697293Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-04-03T16:26:59.4723811Z http.https://github.com/.extraheader 2024-04-03T16:26:59.4735327Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-04-03T16:26:59.4773751Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-04-03T16:26:59.5090224Z Entering 'android/libs/fbjni' 2024-04-03T16:26:59.5124328Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5159922Z Entering 'third_party/FP16' 2024-04-03T16:26:59.5194877Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5230150Z Entering 'third_party/FXdiv' 2024-04-03T16:26:59.5263525Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5298394Z Entering 'third_party/NNPACK' 2024-04-03T16:26:59.5333687Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5369448Z Entering 'third_party/QNNPACK' 2024-04-03T16:26:59.5404887Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5439718Z Entering 'third_party/VulkanMemoryAllocator' 2024-04-03T16:26:59.5472974Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5508313Z Entering 'third_party/XNNPACK' 2024-04-03T16:26:59.5541411Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5594771Z Entering 'third_party/benchmark' 2024-04-03T16:26:59.5628103Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5663184Z Entering 'third_party/cpuinfo' 2024-04-03T16:26:59.5696444Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5732684Z Entering 'third_party/cub' 2024-04-03T16:26:59.5766195Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5801569Z Entering 'third_party/cudnn_frontend' 2024-04-03T16:26:59.5836406Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5870500Z Entering 'third_party/cutlass' 2024-04-03T16:26:59.5903647Z http.https://github.com/.extraheader 2024-04-03T16:26:59.5946426Z Entering 'third_party/eigen' 2024-04-03T16:26:59.5980203Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6017979Z Entering 'third_party/fbgemm' 2024-04-03T16:26:59.6051769Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6087188Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-04-03T16:26:59.6119688Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6155611Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-04-03T16:26:59.6188971Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6223177Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-04-03T16:26:59.6255371Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6297491Z Entering 'third_party/fbgemm/third_party/googletest' 2024-04-03T16:26:59.6330418Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6365889Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-04-03T16:26:59.6397105Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6432744Z Entering 'third_party/flatbuffers' 2024-04-03T16:26:59.6465980Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6503356Z Entering 'third_party/fmt' 2024-04-03T16:26:59.6536064Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6570513Z Entering 'third_party/foxi' 2024-04-03T16:26:59.6603517Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6638407Z Entering 'third_party/gemmlowp/gemmlowp' 2024-04-03T16:26:59.6671598Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6706675Z Entering 'third_party/gloo' 2024-04-03T16:26:59.6740195Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6774535Z Entering 'third_party/googletest' 2024-04-03T16:26:59.6808576Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6844059Z Entering 'third_party/ideep' 2024-04-03T16:26:59.6876977Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6910012Z Entering 'third_party/ideep/mkl-dnn' 2024-04-03T16:26:59.6943665Z http.https://github.com/.extraheader 2024-04-03T16:26:59.6986794Z Entering 'third_party/ios-cmake' 2024-04-03T16:26:59.7020470Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7055078Z Entering 'third_party/ittapi' 2024-04-03T16:26:59.7088021Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7123597Z Entering 'third_party/kineto' 2024-04-03T16:26:59.7156304Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7190791Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-04-03T16:26:59.7224674Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7259918Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-04-03T16:26:59.7292517Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7329535Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-04-03T16:26:59.7362190Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7397172Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-04-03T16:26:59.7430560Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7465690Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-04-03T16:26:59.7498565Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7532154Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-04-03T16:26:59.7564969Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7601310Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-04-03T16:26:59.7633064Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7669068Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-04-03T16:26:59.7702434Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7737892Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-04-03T16:26:59.7770382Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7806733Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-04-03T16:26:59.7839123Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7875623Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-04-03T16:26:59.7907822Z http.https://github.com/.extraheader 2024-04-03T16:26:59.7942959Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-04-03T16:26:59.7974925Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8011337Z Entering 'third_party/mimalloc' 2024-04-03T16:26:59.8045065Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8079895Z Entering 'third_party/nccl/nccl' 2024-04-03T16:26:59.8112141Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8147206Z Entering 'third_party/neon2sse' 2024-04-03T16:26:59.8180772Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8214459Z Entering 'third_party/nlohmann' 2024-04-03T16:26:59.8247261Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8282668Z Entering 'third_party/onnx' 2024-04-03T16:26:59.8315133Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8368539Z Entering 'third_party/onnx/third_party/benchmark' 2024-04-03T16:26:59.8400935Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8435901Z Entering 'third_party/onnx/third_party/pybind11' 2024-04-03T16:26:59.8469023Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8506299Z Entering 'third_party/onnx-tensorrt' 2024-04-03T16:26:59.8540418Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8573696Z Entering 'third_party/onnx-tensorrt/third_party/onnx' 2024-04-03T16:26:59.8606682Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8646469Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/benchmark' 2024-04-03T16:26:59.8679256Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8714940Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11' 2024-04-03T16:26:59.8747187Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8781162Z Entering 'third_party/onnx-tensorrt/third_party/onnx/third_party/pybind11/tools/clang' 2024-04-03T16:26:59.8813945Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8854512Z Entering 'third_party/pocketfft' 2024-04-03T16:26:59.8887184Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8921680Z Entering 'third_party/protobuf' 2024-04-03T16:26:59.8954386Z http.https://github.com/.extraheader 2024-04-03T16:26:59.8992888Z Entering 'third_party/protobuf/third_party/benchmark' 2024-04-03T16:26:59.9025023Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9059783Z Entering 'third_party/protobuf/third_party/googletest' 2024-04-03T16:26:59.9092555Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9129412Z Entering 'third_party/psimd' 2024-04-03T16:26:59.9162696Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9197598Z Entering 'third_party/pthreadpool' 2024-04-03T16:26:59.9231179Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9266050Z Entering 'third_party/pybind11' 2024-04-03T16:26:59.9299886Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9334750Z Entering 'third_party/python-peachpy' 2024-04-03T16:26:59.9367354Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9403132Z Entering 'third_party/sleef' 2024-04-03T16:26:59.9434711Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9470249Z Entering 'third_party/tbb' 2024-04-03T16:26:59.9503212Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9539752Z Entering 'third_party/tensorpipe' 2024-04-03T16:26:59.9572794Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9606885Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-04-03T16:26:59.9638543Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9672996Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-04-03T16:26:59.9705131Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9739299Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-04-03T16:26:59.9772449Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9808129Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-04-03T16:26:59.9839777Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9872396Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-04-03T16:26:59.9904357Z http.https://github.com/.extraheader 2024-04-03T16:26:59.9942780Z Entering 'third_party/zstd' 2024-04-03T16:26:59.9977066Z http.https://github.com/.extraheader 2024-04-03T16:27:00.0343800Z Cleaning up orphan processes